-- Logs begin at Fri 2022-09-09 11:32:43 UTC, end at Fri 2022-09-09 12:32:59 UTC. -- Sep 09 11:58:17.756124 dv-cinder-852211-12-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 09 11:58:17.763652 dv-cinder-852211-12-iscsi devstack@keystone.service[81576]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 09 11:58:17.766203 dv-cinder-852211-12-iscsi devstack@keystone.service[81576]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri Sep 9 11:58:17 2022] *** Sep 09 11:58:17.766337 dv-cinder-852211-12-iscsi devstack@keystone.service[81576]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 09 11:58:17.766459 dv-cinder-852211-12-iscsi devstack@keystone.service[81576]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 09 11:58:17.766566 dv-cinder-852211-12-iscsi devstack@keystone.service[81576]: nodename: dv-cinder-852211-12-iscsi Sep 09 11:58:17.766669 dv-cinder-852211-12-iscsi devstack@keystone.service[81576]: machine: x86_64 Sep 09 11:58:17.766774 dv-cinder-852211-12-iscsi devstack@keystone.service[81576]: clock source: unix Sep 09 11:58:17.766874 dv-cinder-852211-12-iscsi devstack@keystone.service[81576]: pcre jit disabled Sep 09 11:58:17.767002 dv-cinder-852211-12-iscsi devstack@keystone.service[81576]: detected number of CPU cores: 4 Sep 09 11:58:17.767112 dv-cinder-852211-12-iscsi devstack@keystone.service[81576]: current working directory: / Sep 09 11:58:17.767208 dv-cinder-852211-12-iscsi devstack@keystone.service[81576]: detected binary path: /usr/bin/uwsgi-core Sep 09 11:58:17.767315 dv-cinder-852211-12-iscsi devstack@keystone.service[81576]: your processes number limit is 47680 Sep 09 11:58:17.767412 dv-cinder-852211-12-iscsi devstack@keystone.service[81576]: your memory page size is 4096 bytes Sep 09 11:58:17.767516 dv-cinder-852211-12-iscsi devstack@keystone.service[81576]: detected max file descriptor number: 2048 Sep 09 11:58:17.767626 dv-cinder-852211-12-iscsi devstack@keystone.service[81576]: lock engine: pthread robust mutexes Sep 09 11:58:17.767876 dv-cinder-852211-12-iscsi devstack@keystone.service[81576]: thunder lock: enabled Sep 09 11:58:17.768169 dv-cinder-852211-12-iscsi devstack@keystone.service[81576]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 09 11:58:17.768340 dv-cinder-852211-12-iscsi devstack@keystone.service[81576]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 09 11:58:17.807915 dv-cinder-852211-12-iscsi devstack@keystone.service[81576]: Python main interpreter initialized at 0x55dcc00d09c0 Sep 09 11:58:17.807915 dv-cinder-852211-12-iscsi devstack@keystone.service[81576]: python threads support enabled Sep 09 11:58:17.807915 dv-cinder-852211-12-iscsi devstack@keystone.service[81576]: your server socket listen backlog is limited to 100 connections Sep 09 11:58:17.807915 dv-cinder-852211-12-iscsi devstack@keystone.service[81576]: your mercy for graceful operations on workers is 90 seconds Sep 09 11:58:17.808282 dv-cinder-852211-12-iscsi devstack@keystone.service[81576]: mapped 403077 bytes (393 KB) for 2 cores Sep 09 11:58:17.808364 dv-cinder-852211-12-iscsi devstack@keystone.service[81576]: *** Operational MODE: preforking *** Sep 09 11:58:17.808601 dv-cinder-852211-12-iscsi devstack@keystone.service[81576]: *** uWSGI is running in multiple interpreter mode *** Sep 09 11:58:17.808601 dv-cinder-852211-12-iscsi devstack@keystone.service[81576]: spawned uWSGI master process (pid: 81576) Sep 09 11:58:17.808800 dv-cinder-852211-12-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 09 11:58:17.809390 dv-cinder-852211-12-iscsi devstack@keystone.service[81576]: spawned uWSGI worker 1 (pid: 81590, cores: 1) Sep 09 11:58:17.809669 dv-cinder-852211-12-iscsi devstack@keystone.service[81576]: spawned uWSGI worker 2 (pid: 81591, cores: 1) Sep 09 11:58:17.809824 dv-cinder-852211-12-iscsi devstack@keystone.service[81576]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 09 11:58:22.128039 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 09 11:58:22.128039 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: from cryptography.utils import int_from_bytes Sep 09 11:58:22.128825 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 09 11:58:22.128825 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: from cryptography.utils import int_from_bytes Sep 09 11:58:22.128942 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 09 11:58:22.128942 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: from cryptography.utils import int_from_bytes Sep 09 11:58:22.129344 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 09 11:58:22.129344 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: from cryptography.utils import int_from_bytes Sep 09 11:58:22.180157 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 09 11:58:22.180308 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 09 11:58:22.180396 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 09 11:58:22.180470 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 09 11:58:22.180535 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] command line args: [] {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 09 11:58:22.180593 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] command line args: [] {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 09 11:58:22.180773 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 09 11:58:22.180865 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 09 11:58:22.180966 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] ================================================================================ {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 09 11:58:22.181090 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] ================================================================================ {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 09 11:58:22.181213 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] admin_token = **** {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:58:22.181300 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] admin_token = **** {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:58:22.181462 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] config_dir = [] {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:58:22.181542 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] config_dir = [] {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:58:22.181676 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:58:22.181779 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:58:22.181856 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] config_source = [] {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:58:22.181932 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] config_source = [] {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:58:22.182013 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] debug = True {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:58:22.182085 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] debug = True {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:58:22.182183 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:58:22.182456 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:58:22.182556 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:58:22.182617 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:58:22.182751 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:58:22.182835 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:58:22.182990 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:58:22.183082 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:58:22.183241 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:58:22.183329 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:58:22.183473 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] list_limit = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:58:22.183551 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] list_limit = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:58:22.183652 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] log_config_append = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:58:22.183731 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] log_config_append = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:58:22.183804 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:58:22.183864 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:58:22.183980 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] log_dir = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:58:22.184059 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] log_dir = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:58:22.184158 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] log_file = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:58:22.184235 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] log_file = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:58:22.184389 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:58:22.184469 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:58:22.184636 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:58:22.184720 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:58:22.184882 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:58:22.184966 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:58:22.185129 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:58:22.185210 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:58:22.185307 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:58:22.185391 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:58:22.185468 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:58:22.185546 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:58:22.185718 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:58:22.185804 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:58:22.185963 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:58:22.186043 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:58:22.186198 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:58:22.186291 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:58:22.186430 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:58:22.186506 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:58:22.186662 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:58:22.186737 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:58:22.187166 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:58:22.187264 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:58:22.187421 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:58:22.187500 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:58:22.187669 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:58:22.187751 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:58:22.187940 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:58:22.188026 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:58:22.188426 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.49/identity {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:58:22.188504 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.49/identity {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:58:22.188629 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] publish_errors = False {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:58:22.188704 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] publish_errors = False {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:58:22.188826 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:58:22.188905 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:58:22.189042 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:58:22.189139 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:58:22.189213 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:58:22.189288 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:58:22.189451 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:58:22.189551 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:58:22.189687 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:58:22.189764 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:58:22.189921 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] standard_threads = False {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:58:22.190002 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] standard_threads = False {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:58:22.190152 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:58:22.190237 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:58:22.190395 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:58:22.190471 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:58:22.190569 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:58:22.190645 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:58:22.190744 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] use_journal = False {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:58:22.190870 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] use_journal = False {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:58:22.190966 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] use_json = False {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:58:22.191037 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] use_json = False {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:58:22.191117 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] use_stderr = False {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:58:22.191194 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] use_stderr = False {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:58:22.191265 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] use_syslog = False {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:58:22.191341 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] use_syslog = False {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:58:22.191489 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:58:22.191573 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 11:58:22.191746 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.191825 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.191987 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.192074 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.192249 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.192331 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.192551 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.193165 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.193316 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.193406 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.193562 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.193746 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.193824 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.194007 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.194086 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.194251 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.194330 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.194503 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.194588 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.194754 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.194836 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.195025 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.195113 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.195264 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.195342 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.195504 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.195582 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.195745 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.195829 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.195993 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.196078 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.196234 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.196310 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.196474 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.196555 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.196733 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.196816 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.196982 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.197061 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.197252 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.197357 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.197453 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.197527 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.197699 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.197790 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.197954 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.198032 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.198206 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.198291 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.198441 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.198538 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.198676 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.198756 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.198945 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.199027 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.199163 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.199254 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.199403 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.199482 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.199649 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.199727 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.199883 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.199969 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.200116 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.200196 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.200347 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.200429 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.200599 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.200684 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.200839 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.200917 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.201077 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.201153 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.201310 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.201395 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.201550 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.201628 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.201789 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.201865 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.202019 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.202103 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.202252 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.202329 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.202481 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.202558 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.202711 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.202863 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.202958 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.203074 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.203187 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.203262 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.203427 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.203521 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.203658 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.203735 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.203888 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.203970 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.204120 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.204232 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.204355 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.204459 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.204592 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.204701 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.204824 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.204947 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.205047 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.205124 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.205289 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.205370 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.205525 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.205654 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.205763 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.205840 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.205998 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.206090 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.206232 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.206312 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.206466 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.206552 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.206707 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.206784 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.206967 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.207088 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.207201 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.207293 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.207434 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.207523 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.207689 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.207769 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.207924 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.208007 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.208186 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.208263 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.208535 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.208783 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.208882 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.208957 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.209076 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.209151 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.209266 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.209341 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.209505 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.209593 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.209745 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.209823 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.209982 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.210063 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.210220 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.210300 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.210470 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.210556 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.210712 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.210818 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.210993 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.211071 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.211202 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.211289 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.211439 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.211523 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.211682 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.211762 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.211921 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.211998 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.212157 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.212236 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.212392 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.212477 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.212627 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.212704 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.212863 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.212941 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.213103 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.213183 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.213348 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.213432 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.213593 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.213670 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.213830 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.213909 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.214069 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.214154 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.214308 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.214386 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.214534 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.214609 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.214770 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.214849 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.215033 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.215127 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.215267 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.215354 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.215498 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.215579 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.215730 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.215808 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.215961 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.216046 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.216191 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.216269 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.216420 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.216498 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.216657 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.216743 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.216895 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.216974 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.217134 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.217212 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.217376 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.217465 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.217637 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.217717 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.217877 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.217963 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.218110 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] auth.external = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.218203 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.218342 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.218430 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] auth.external = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.218572 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.218651 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.218859 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.218991 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.219118 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] auth.password = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.219207 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.219349 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] auth.password = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.219428 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] auth.token = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.219583 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] auth.token = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.219662 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.219816 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.219902 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.220052 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.220131 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.220285 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.220363 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.220523 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.220611 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.220769 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.220847 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.221001 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.221080 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.221238 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.221327 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] credential.caching = True {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.221482 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] credential.caching = True {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.221562 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.221734 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.221814 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.221981 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.222067 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.222226 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.222306 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.222472 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.222551 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.222707 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.222796 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.222963 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.223047 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.223217 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.223297 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.223452 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.223543 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.223722 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.223806 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.224041 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.224233 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.224311 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.224391 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.224513 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.224708 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.224787 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.224980 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.225059 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.225229 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] federation.caching = True {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.225314 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.225470 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.225550 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.225717 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] federation.caching = True {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.225795 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.225962 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.226043 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.226193 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.226271 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.226442 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.226530 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.226678 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.226756 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.226953 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.227032 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.227198 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.227289 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.227456 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.227534 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.227694 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.227772 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.227945 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.228031 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.228182 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] identity.caching = True {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.228262 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.228432 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.228510 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.228673 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] identity.caching = True {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.228830 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.228906 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.228993 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.229133 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.229212 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.229382 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.229469 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.229616 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.229695 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.229856 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.229935 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.230089 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.230176 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.230332 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.230411 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.230567 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.230645 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.230800 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.230887 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.231057 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.231137 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.231294 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.231372 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.231534 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.231625 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.231782 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.231861 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.232032 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.232111 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.232277 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.232375 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.232533 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.232612 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.232777 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.232855 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.233017 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.233104 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.233250 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.233329 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.233489 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.233568 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.233721 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.233809 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.233959 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.234038 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.234191 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.234274 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.234428 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.234514 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.234668 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.234747 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.234943 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.235026 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.235167 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.235258 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.235400 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.235478 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.235640 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.235717 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.235887 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.235976 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.236126 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.236206 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.236354 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.236432 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.236592 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.236673 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.236821 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.236900 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.237059 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.237136 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.237292 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.237379 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.237529 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.237608 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.237766 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.237844 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.238010 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.238117 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.238253 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.238375 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.238607 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.238686 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.238849 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.238947 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.239122 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.239201 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.239879 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.239976 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.240149 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.240223 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.240393 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.240475 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.240630 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.240700 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.240871 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.240953 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.241106 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] ldap.user = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.241193 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.241345 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.241424 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.241583 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] ldap.user = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.241664 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.241829 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.241907 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.242081 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.242161 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.242313 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.242394 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.242566 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.242646 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.242800 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.242879 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.243058 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.243141 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.243296 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.243375 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.243527 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.243606 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.243765 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.243845 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.243997 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.244078 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.244234 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.244318 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.244468 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.244549 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.244717 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.244795 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.244964 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.245042 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.245209 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.245289 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.245441 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.245526 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.245690 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.245768 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.245923 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.246004 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.246159 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.246236 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.246397 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.246476 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.246635 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.246715 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.246871 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.246975 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.247140 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.247227 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.247377 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.247457 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.247629 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.247708 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.247860 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.247938 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.248104 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.248189 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.248337 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.248422 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.248571 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.248649 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.248810 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.248891 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.249045 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.249123 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.249286 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.249364 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.249519 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.249600 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.249757 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.249836 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.249989 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.250067 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] resource.caching = True {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.250227 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.250309 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.250471 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] resource.caching = True {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.250552 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.250707 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.250785 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.250980 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.251067 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.251219 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.251298 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.251459 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.251538 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.251715 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.251797 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.251951 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.252030 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.252189 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.252339 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.252415 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.252485 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] role.caching = True {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.252653 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.252733 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] role.driver = sql {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.252883 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.252970 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] role.caching = True {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.253129 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] role.driver = sql {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.253208 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.253362 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.253441 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.253610 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.253696 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.253858 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.253937 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.254104 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.254182 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.254349 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.254436 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.254593 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.254672 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.254840 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.254932 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.255690 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.255797 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.255946 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.256032 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.256198 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.256278 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.256449 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.256538 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.256699 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.256779 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.256987 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.257094 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.257239 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.257327 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.257572 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.257651 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.257826 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.257905 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.258077 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.258164 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.258328 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.258408 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.258564 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.258643 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.258815 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.258949 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.259078 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.259159 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.259324 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.259402 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.259559 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.259640 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.259792 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.259872 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.260026 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.260105 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.260259 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.260345 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.260492 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.260572 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.260745 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.260825 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.260991 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.261071 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.261232 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.261311 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.261465 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.261545 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.261706 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.261785 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] token.caching = True {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.261939 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.262019 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.262177 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] token.caching = True {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.262256 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.262413 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.262492 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.262662 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.262740 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.262915 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.263002 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.263164 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.263245 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.263410 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.263490 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.263649 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.263727 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.263895 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.263977 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.264133 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.264212 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.264373 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.264453 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.264610 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.264697 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.264847 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.264925 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.265084 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.265164 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.265332 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.265412 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.265572 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.265660 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.265831 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.265918 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.266077 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.266158 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] database.connection = **** {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.266320 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.266406 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.266568 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.266648 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] database.connection = **** {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.266808 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.266887 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.267085 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.267180 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.267321 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.267400 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.267560 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.267640 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.267794 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.267880 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.268028 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.268107 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.268263 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.268343 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.268498 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.268585 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.268742 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.268821 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.268977 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.269056 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.269238 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.269327 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.269456 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.269535 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.269702 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.269782 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.269933 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.270021 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.270174 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.270256 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.270406 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.270485 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.270557 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81590) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 09 11:58:22.270719 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.270964 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 11:58:22.271121 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 09 11:58:22.299719 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81590) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:58:22.299952 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81590) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:58:22.300054 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81591) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:58:22.300179 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81590) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:58:22.300266 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81591) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:58:22.300415 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81591) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:58:22.309117 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81590) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:58:22.309363 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81591) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:58:22.338132 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81590) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:58:22.338364 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81590) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:58:22.339968 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81591) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:58:22.340202 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81591) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:58:22.392497 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81591) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:58:22.392639 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81590) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:58:22.441836 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81591) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:58:22.442069 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81591) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:58:22.442393 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81590) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:58:22.443241 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81590) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:58:22.600013 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81590) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:58:22.600251 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81590) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:58:22.600343 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81591) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:58:22.600468 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81590) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:58:22.600554 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81591) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:58:22.600689 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81590) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:58:22.600776 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81591) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:58:22.600906 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81590) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:58:22.600989 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81591) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:58:22.601120 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81590) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:58:22.601213 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81591) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:58:22.601338 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81591) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:58:22.624292 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81590) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:58:22.624525 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81590) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:58:22.624668 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81591) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:58:22.624795 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81590) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:58:22.624911 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81591) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:58:22.625016 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81590) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:58:22.627490 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81591) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:58:22.627571 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81590) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:58:22.627635 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81591) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:58:22.627712 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81590) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:58:22.627777 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81591) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:58:22.627836 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81590) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:58:22.627897 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81591) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:58:22.627897 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81591) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:58:22.637639 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81590) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:58:22.638123 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81591) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:58:22.651444 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81590) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:58:22.651559 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81591) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:58:22.651679 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81590) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:58:22.651805 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81591) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:58:22.651893 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81590) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:58:22.652014 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81591) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:58:22.652102 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81590) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:58:22.652240 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81591) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:58:22.652334 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81590) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:58:22.652457 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81591) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:58:22.652540 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81590) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:58:22.652671 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81591) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:58:22.652753 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81590) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:58:22.652884 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81591) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 11:58:22.669708 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.669922 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 09 11:58:22.670165 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.670274 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.670381 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 09 11:58:22.670478 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 09 11:58:22.670728 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.670811 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.670969 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 09 11:58:22.671051 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.671233 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.671318 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.671497 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.671579 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.671757 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.672014 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.681148 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.681431 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.681646 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.681729 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.681926 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.682022 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.682199 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.682483 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.689216 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81590) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 11:58:22.689692 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81591) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 11:58:22.692126 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81590) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 11:58:22.692416 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.693147 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81591) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 11:58:22.693297 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.693436 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.693565 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.693781 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.693891 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.694044 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.694182 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.694313 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.694472 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.694594 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.694739 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.694867 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.695037 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.695167 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.695318 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.695437 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.695591 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.695867 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.696141 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.718709 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.719366 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.720205 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81590) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 11:58:22.720493 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.720841 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81591) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 11:58:22.721127 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.725336 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81590) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 11:58:22.725620 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.725840 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81591) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 11:58:22.725923 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.726128 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.726414 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.732201 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81590) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 11:58:22.732490 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.732788 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81591) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 11:58:22.733078 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.738128 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81590) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 11:58:22.738432 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.738710 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81591) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 11:58:22.738794 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.739024 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.739120 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.739310 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.739437 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.739616 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.739697 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.739894 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.739978 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.740166 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.740250 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.740437 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.740707 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.756179 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.756454 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.756633 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.756916 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.760319 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81590) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 11:58:22.760798 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81591) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 11:58:22.763581 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.763882 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.764091 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.764392 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.768093 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81590) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 11:58:22.768575 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81591) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 11:58:22.771906 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81590) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 11:58:22.772396 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81591) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 11:58:22.775148 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.775452 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.775663 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.775747 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.775966 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.776048 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.776256 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.776340 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.776543 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.776632 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.776826 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.777118 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.792424 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81590) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 11:58:22.792718 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.792952 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81591) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 11:58:22.793036 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.793242 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.793337 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.793511 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.793775 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.800075 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.800627 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.801624 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.802332 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.803300 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.803582 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.803929 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.804209 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.806306 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81590) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 11:58:22.806593 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.806880 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.806987 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81591) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 11:58:22.807191 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.807285 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.807492 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.807575 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.807777 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.808175 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.819457 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81590) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 11:58:22.819744 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.820029 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.820238 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81591) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 11:58:22.820322 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.820527 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.820681 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.820811 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.820970 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.821097 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.821251 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.821481 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.821764 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.822057 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.837165 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81590) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 11:58:22.838132 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81591) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 11:58:22.840102 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81590) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 11:58:22.841094 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81591) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 11:58:22.842998 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.844004 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.844608 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.845773 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.846084 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81590) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 11:58:22.846366 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.846763 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.847281 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81591) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 11:58:22.847567 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.847964 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.853143 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.854510 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.854596 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81590) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 11:58:22.855994 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81591) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 11:58:22.857502 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.857904 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.858195 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.858473 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.858894 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.859316 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.859606 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.859888 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.866527 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81590) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 11:58:22.866820 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.867129 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.867716 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81591) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 11:58:22.868007 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.868298 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.873972 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81590) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 11:58:22.874258 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.874549 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.874825 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.875135 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.875218 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81591) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 11:58:22.875411 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.875505 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.875686 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.875770 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.875977 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.876059 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.876248 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.876386 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.876521 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.876605 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.876792 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.876875 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.877065 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.877147 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.877333 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.877415 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.877611 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.877694 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81590) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.877886 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.878151 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.878432 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.878804 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81591) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 11:58:22.912480 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55dcc00d09c0 pid: 81590 (default app) Sep 09 11:58:22.913830 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55dcc00d09c0 pid: 81591 (default app) Sep 09 11:58:22.915772 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb39d33-9bf9-4dd3-bb5a-29157b9fd32d None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:22.916060 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb39d33-9bf9-4dd3-bb5a-29157b9fd32d None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:22.916332 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb39d33-9bf9-4dd3-bb5a-29157b9fd32d None None] PATH_INFO: `/v3` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:22.920495 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1/1] 10.222.0.49 () {50 vars in 791 bytes} [Fri Sep 9 11:58:22 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:58:29.687604 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1b339d-feed-47c7-8800-90a9d6563dea None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:29.688075 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1b339d-feed-47c7-8800-90a9d6563dea None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:29.688183 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1b339d-feed-47c7-8800-90a9d6563dea None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:29.692421 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1/2] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 11:58:29 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:58:29.698230 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70ece05-092a-4d17-94b9-085e65d750ab None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:29.698656 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70ece05-092a-4d17-94b9-085e65d750ab None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:29.698793 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70ece05-092a-4d17-94b9-085e65d750ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:29.769320 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: INFO dbcounter [None req-c70ece05-092a-4d17-94b9-085e65d750ab None None] Registered counter for database keystone Sep 09 11:58:29.788498 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG oslo_db.sqlalchemy.engines [None req-c70ece05-092a-4d17-94b9-085e65d750ab None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81590) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 09 11:58:29.794708 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writer thread running {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 09 11:58:29.972028 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG passlib.handlers.bcrypt [None req-c70ece05-092a-4d17-94b9-085e65d750ab None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81590) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 09 11:58:29.972339 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG passlib.handlers.bcrypt [None req-c70ece05-092a-4d17-94b9-085e65d750ab None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81590) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 09 11:58:30.029997 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-c70ece05-092a-4d17-94b9-085e65d750ab None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:58:30.031606 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-c70ece05-092a-4d17-94b9-085e65d750ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:30.163400 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 2/3] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 11:58:29 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 466 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Sep 09 11:58:30.607339 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1b339d-feed-47c7-8800-90a9d6563dea None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:30.676597 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: INFO dbcounter [None req-3b1b339d-feed-47c7-8800-90a9d6563dea None None] Registered counter for database keystone Sep 09 11:58:30.699186 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG oslo_db.sqlalchemy.engines [None req-3b1b339d-feed-47c7-8800-90a9d6563dea None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81591) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 09 11:58:30.705246 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writer thread running {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 09 11:58:30.824368 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1b339d-feed-47c7-8800-90a9d6563dea None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:30.828544 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bc8469-1745-4734-bf11-489efbd84c9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:30.829164 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bc8469-1745-4734-bf11-489efbd84c9e admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:30.829448 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bc8469-1745-4734-bf11-489efbd84c9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:30.829717 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bc8469-1745-4734-bf11-489efbd84c9e admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:30.838692 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bc8469-1745-4734-bf11-489efbd84c9e admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:30.889703 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: WARNING keystone.common.rbac_enforcer.enforcer [None req-b8bc8469-1745-4734-bf11-489efbd84c9e admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 09 11:58:30.889985 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bc8469-1745-4734-bf11-489efbd84c9e admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:30.900400 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 2/4] 10.222.0.49 () {58 vars in 1324 bytes} [Fri Sep 9 11:58:30 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 11:58:30.906336 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70ece05-092a-4d17-94b9-085e65d750ab None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:30.923382 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70ece05-092a-4d17-94b9-085e65d750ab None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:30.927972 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedf3dc6-7b40-43df-ae0d-a4412c0319b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:30.929069 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedf3dc6-7b40-43df-ae0d-a4412c0319b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:30.929349 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedf3dc6-7b40-43df-ae0d-a4412c0319b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:30.929626 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedf3dc6-7b40-43df-ae0d-a4412c0319b7 admin admin] PATH_INFO: `/v3/projects/ae50911c7cf74f75b5b41fa5275c8468` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:30.931203 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedf3dc6-7b40-43df-ae0d-a4412c0319b7 admin admin] RBAC: Authorizing `identity:get_project(project_id=ae50911c7cf74f75b5b41fa5275c8468)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:30.984732 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: WARNING keystone.common.rbac_enforcer.enforcer [None req-bedf3dc6-7b40-43df-ae0d-a4412c0319b7 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 09 11:58:30.985070 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedf3dc6-7b40-43df-ae0d-a4412c0319b7 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:30.989547 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 3/5] 10.222.0.49 () {56 vars in 1207 bytes} [Fri Sep 9 11:58:30 2022] GET /identity/v3/projects/ae50911c7cf74f75b5b41fa5275c8468 => generated 344 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:58:33.123924 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce4e7e7-8982-4eed-999a-3b42aae32487 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:33.124447 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce4e7e7-8982-4eed-999a-3b42aae32487 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:33.124447 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce4e7e7-8982-4eed-999a-3b42aae32487 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:33.128579 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 3/6] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 11:58:33 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:58:33.134948 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afd9bed-0dd8-4a0c-88b9-2011c7411fad None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:33.135344 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afd9bed-0dd8-4a0c-88b9-2011c7411fad None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:33.135499 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afd9bed-0dd8-4a0c-88b9-2011c7411fad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:33.155684 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-6afd9bed-0dd8-4a0c-88b9-2011c7411fad None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:58:33.157204 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-6afd9bed-0dd8-4a0c-88b9-2011c7411fad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:33.164965 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 4/7] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 11:58:33 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 09 11:58:33.433661 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce4e7e7-8982-4eed-999a-3b42aae32487 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:33.446257 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce4e7e7-8982-4eed-999a-3b42aae32487 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:33.450484 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649310e8-2354-481b-9769-6a616df8aa93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:33.451125 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649310e8-2354-481b-9769-6a616df8aa93 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:33.451408 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649310e8-2354-481b-9769-6a616df8aa93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:33.451681 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649310e8-2354-481b-9769-6a616df8aa93 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:33.460052 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649310e8-2354-481b-9769-6a616df8aa93 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:33.461829 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649310e8-2354-481b-9769-6a616df8aa93 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:33.470514 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 4/8] 10.222.0.49 () {58 vars in 1324 bytes} [Fri Sep 9 11:58:33 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 11:58:33.476392 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afd9bed-0dd8-4a0c-88b9-2011c7411fad None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:33.485360 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afd9bed-0dd8-4a0c-88b9-2011c7411fad None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:33.489755 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5812fa16-3fce-4aac-8a82-76b8b782480c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:33.490482 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5812fa16-3fce-4aac-8a82-76b8b782480c admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:33.490760 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5812fa16-3fce-4aac-8a82-76b8b782480c admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:33.491072 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5812fa16-3fce-4aac-8a82-76b8b782480c admin admin] PATH_INFO: `/v3/users/136011715ec34418adc0f536f6c7357c` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:33.492380 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5812fa16-3fce-4aac-8a82-76b8b782480c admin admin] RBAC: Authorizing `identity:get_user(user_id=136011715ec34418adc0f536f6c7357c)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:33.494191 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5812fa16-3fce-4aac-8a82-76b8b782480c admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:33.495746 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 5/9] 10.222.0.49 () {56 vars in 1198 bytes} [Fri Sep 9 11:58:33 2022] GET /identity/v3/users/136011715ec34418adc0f536f6c7357c => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:58:35.721555 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aca6599-45df-4fe6-b529-17695e85a8a8 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:35.722113 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aca6599-45df-4fe6-b529-17695e85a8a8 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:35.722113 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aca6599-45df-4fe6-b529-17695e85a8a8 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:35.723566 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 5/10] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 11:58:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:58:35.729029 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1572e27-4597-4bd9-8196-2ab8f32b26f4 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:35.729429 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1572e27-4597-4bd9-8196-2ab8f32b26f4 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:35.729590 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1572e27-4597-4bd9-8196-2ab8f32b26f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:35.747893 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-e1572e27-4597-4bd9-8196-2ab8f32b26f4 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:58:35.749066 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-e1572e27-4597-4bd9-8196-2ab8f32b26f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:35.775587 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a987cf-771d-4c53-b7e6-6b0da10f388b None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:35.775874 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a987cf-771d-4c53-b7e6-6b0da10f388b None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:35.776148 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a987cf-771d-4c53-b7e6-6b0da10f388b None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:35.777465 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 6/11] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 11:58:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 11:58:35.782497 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9177cbf9-5ab0-41c2-8915-bcea145b4c2b None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:35.782778 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9177cbf9-5ab0-41c2-8915-bcea145b4c2b None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:35.783077 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9177cbf9-5ab0-41c2-8915-bcea145b4c2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:35.826317 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG passlib.handlers.bcrypt [None req-9177cbf9-5ab0-41c2-8915-bcea145b4c2b None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81591) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 09 11:58:35.826706 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG passlib.handlers.bcrypt [None req-9177cbf9-5ab0-41c2-8915-bcea145b4c2b None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81591) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 09 11:58:35.830950 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 6/12] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 11:58:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:58:35.875539 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-9177cbf9-5ab0-41c2-8915-bcea145b4c2b None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:58:35.877127 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-9177cbf9-5ab0-41c2-8915-bcea145b4c2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:35.977188 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 7/13] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 11:58:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 11:58:36.099060 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1572e27-4597-4bd9-8196-2ab8f32b26f4 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:36.123286 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:36.203341 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1572e27-4597-4bd9-8196-2ab8f32b26f4 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:36.236414 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78ee874-bac1-4291-9798-21e9ebc49d45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:36.236983 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78ee874-bac1-4291-9798-21e9ebc49d45 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:36.237256 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78ee874-bac1-4291-9798-21e9ebc49d45 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:36.237524 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78ee874-bac1-4291-9798-21e9ebc49d45 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:36.248638 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78ee874-bac1-4291-9798-21e9ebc49d45 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:58:36.258321 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-f78ee874-bac1-4291-9798-21e9ebc49d45 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:58:36.258321 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:58:36.258321 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:58:36.258321 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:58:36.258321 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:58:36.258321 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:58:36.258321 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:58:36.258321 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:58:36.258321 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:58:36.258321 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:58:36.258321 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:58:36.258321 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:58:36.258321 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:58:36.258321 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:58:36.258321 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:58:36.258321 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:58:36.259452 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:58:36.259452 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:58:36.259452 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:58:36.259452 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:58:36.259452 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:58:36.259452 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:58:36.259452 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:58:36.259452 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:58:36.259452 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:58:36.259452 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:58:36.259452 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:58:36.259452 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 11:58:36.259452 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:58:36.259452 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:58:36.259452 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:58:36.259452 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:58:36.260411 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:58:36.260411 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:58:36.260411 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:58:36.260411 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:58:36.260411 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:58:36.260411 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:58:36.260411 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:58:36.260411 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:58:36.260411 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:58:36.260411 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:58:36.260411 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:58:36.260411 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 11:58:36.261174 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9177cbf9-5ab0-41c2-8915-bcea145b4c2b None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:36.261263 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 7/14] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Sep 9 11:58:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:58:36.265134 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78ee874-bac1-4291-9798-21e9ebc49d45 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:36.346200 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:36.362764 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9177cbf9-5ab0-41c2-8915-bcea145b4c2b None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:36.365622 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:36.366264 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78ee874-bac1-4291-9798-21e9ebc49d45 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:36.401376 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7a23bd-05e2-48c7-8e64-4094136a4b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:36.401809 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f5c7d2-55ac-4384-9669-52b680323814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:36.402235 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7a23bd-05e2-48c7-8e64-4094136a4b0c None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:36.402411 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f5c7d2-55ac-4384-9669-52b680323814 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:36.402525 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7a23bd-05e2-48c7-8e64-4094136a4b0c None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:36.402688 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f5c7d2-55ac-4384-9669-52b680323814 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:36.402784 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7a23bd-05e2-48c7-8e64-4094136a4b0c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:36.402987 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f5c7d2-55ac-4384-9669-52b680323814 None admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:36.403645 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f5c7d2-55ac-4384-9669-52b680323814 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:58:36.404054 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f5c7d2-55ac-4384-9669-52b680323814 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:36.405454 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f5c7d2-55ac-4384-9669-52b680323814 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:36.417272 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 8/15] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 9 11:58:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:58:36.422881 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f5c7d2-55ac-4384-9669-52b680323814 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:36.472433 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7a23bd-05e2-48c7-8e64-4094136a4b0c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:36.474579 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7a23bd-05e2-48c7-8e64-4094136a4b0c None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:36.523448 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f5c7d2-55ac-4384-9669-52b680323814 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:36.556625 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef1689f-367c-4f1b-a817-2bec92fb4092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:36.557250 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef1689f-367c-4f1b-a817-2bec92fb4092 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:36.557527 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef1689f-367c-4f1b-a817-2bec92fb4092 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:36.557795 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef1689f-367c-4f1b-a817-2bec92fb4092 None admin] PATH_INFO: `/v3/users/136011715ec34418adc0f536f6c7357c` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:36.558982 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef1689f-367c-4f1b-a817-2bec92fb4092 None admin] RBAC: Authorizing `identity:get_user(user_id=136011715ec34418adc0f536f6c7357c)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:36.560430 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef1689f-367c-4f1b-a817-2bec92fb4092 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:36.561989 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 9/16] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Sep 9 11:58:36 2022] GET /identity/v3/users/136011715ec34418adc0f536f6c7357c => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:58:36.567812 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef1689f-367c-4f1b-a817-2bec92fb4092 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:36.574662 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 8/17] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 9 11:58:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 09 11:58:36.579988 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7a23bd-05e2-48c7-8e64-4094136a4b0c None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:36.582300 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:36.582676 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:36.672098 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef1689f-367c-4f1b-a817-2bec92fb4092 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:36.681100 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7a23bd-05e2-48c7-8e64-4094136a4b0c None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:36.707449 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617d66aa-f332-4a6e-8ea4-220c800c6e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:36.708067 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617d66aa-f332-4a6e-8ea4-220c800c6e22 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:36.708346 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617d66aa-f332-4a6e-8ea4-220c800c6e22 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:36.709107 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617d66aa-f332-4a6e-8ea4-220c800c6e22 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:36.710231 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617d66aa-f332-4a6e-8ea4-220c800c6e22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:36.711701 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617d66aa-f332-4a6e-8ea4-220c800c6e22 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:36.713226 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 10/18] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 11:58:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:58:36.714866 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67ed662-c681-4fd8-993e-a38ce9a7f9d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:36.715741 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67ed662-c681-4fd8-993e-a38ce9a7f9d9 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:36.716020 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67ed662-c681-4fd8-993e-a38ce9a7f9d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:36.716291 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67ed662-c681-4fd8-993e-a38ce9a7f9d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:36.718999 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617d66aa-f332-4a6e-8ea4-220c800c6e22 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:36.735675 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67ed662-c681-4fd8-993e-a38ce9a7f9d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:36.737173 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67ed662-c681-4fd8-993e-a38ce9a7f9d9 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:36.756767 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-e67ed662-c681-4fd8-993e-a38ce9a7f9d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:58:36.756767 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:58:36.756767 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:58:36.756767 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:58:36.756767 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:58:36.756767 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:58:36.756767 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:58:36.756767 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:58:36.756767 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:58:36.756767 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:58:36.756767 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:58:36.756767 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:58:36.756767 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:58:36.756767 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:58:36.756767 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:58:36.756767 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:58:36.757816 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:58:36.757816 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:58:36.757816 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:58:36.757816 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:58:36.757816 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:58:36.757816 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:58:36.757816 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:58:36.757816 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 11:58:36.757816 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:58:36.757816 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:58:36.757816 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:58:36.757816 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:58:36.757816 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:58:36.757816 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:58:36.757816 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:58:36.757816 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:58:36.757816 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:58:36.763266 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:58:36.763266 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:58:36.763266 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 11:58:36.763266 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 9/19] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 11:58:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:58:36.763754 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67ed662-c681-4fd8-993e-a38ce9a7f9d9 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:36.801742 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:36.820028 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:36.823394 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617d66aa-f332-4a6e-8ea4-220c800c6e22 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:36.860068 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a83f41-c8b9-4f77-ae26-da06eeed4b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:36.860693 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a83f41-c8b9-4f77-ae26-da06eeed4b00 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:36.860975 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a83f41-c8b9-4f77-ae26-da06eeed4b00 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:36.861245 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a83f41-c8b9-4f77-ae26-da06eeed4b00 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:36.861958 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a83f41-c8b9-4f77-ae26-da06eeed4b00 None admin] RBAC: Adding query filter params (role.id=3f2ca33902a04b6dbc2ef7d06d7289d3, scope.domain.id=default, user.id=136011715ec34418adc0f536f6c7357c) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:58:36.862261 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a83f41-c8b9-4f77-ae26-da06eeed4b00 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:36.863726 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a83f41-c8b9-4f77-ae26-da06eeed4b00 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:36.869317 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67ed662-c681-4fd8-993e-a38ce9a7f9d9 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:36.870980 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 11/20] 10.222.0.49 () {56 vars in 1322 bytes} [Fri Sep 9 11:58:36 2022] GET /identity/v3/role_assignments?user.id=136011715ec34418adc0f536f6c7357c&scope.domain.id=default&role.id=3f2ca33902a04b6dbc2ef7d06d7289d3 => generated 233 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:58:36.904256 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e43136-1421-4579-a3d1-e621d32abb2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:36.904885 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e43136-1421-4579-a3d1-e621d32abb2f None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:36.905164 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e43136-1421-4579-a3d1-e621d32abb2f None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:36.905433 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e43136-1421-4579-a3d1-e621d32abb2f None admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:36.906086 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e43136-1421-4579-a3d1-e621d32abb2f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:58:36.906388 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e43136-1421-4579-a3d1-e621d32abb2f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:36.907853 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e43136-1421-4579-a3d1-e621d32abb2f None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:36.938934 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 10/21] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 9 11:58:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:58:39.373401 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f06cac9-6006-4b22-b0da-05a4fa3e3d42 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:39.374443 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f06cac9-6006-4b22-b0da-05a4fa3e3d42 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:39.374949 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f06cac9-6006-4b22-b0da-05a4fa3e3d42 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:39.376627 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 12/22] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 11:58:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:58:39.383684 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562a6ad9-e715-4f93-95c4-1f58a4b2bbca None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:39.384216 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562a6ad9-e715-4f93-95c4-1f58a4b2bbca None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:39.384216 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562a6ad9-e715-4f93-95c4-1f58a4b2bbca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:39.410410 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-562a6ad9-e715-4f93-95c4-1f58a4b2bbca None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:58:39.411676 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-562a6ad9-e715-4f93-95c4-1f58a4b2bbca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:39.503165 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 11/23] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 11:58:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:58:39.541734 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a74008-b224-4353-a067-f5e369b8844b None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:39.542431 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a74008-b224-4353-a067-f5e369b8844b None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:39.542928 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a74008-b224-4353-a067-f5e369b8844b None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:39.544748 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 13/24] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 11:58:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:58:39.549694 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfa1bb1-8886-4f19-b319-6b641646b8f9 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:39.549981 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfa1bb1-8886-4f19-b319-6b641646b8f9 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:39.550256 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfa1bb1-8886-4f19-b319-6b641646b8f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:39.579301 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-ecfa1bb1-8886-4f19-b319-6b641646b8f9 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:58:39.580506 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-ecfa1bb1-8886-4f19-b319-6b641646b8f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:39.608730 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3bf03e-289c-4cdb-b2f9-38cf5cf7d6d5 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:39.609364 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3bf03e-289c-4cdb-b2f9-38cf5cf7d6d5 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:39.609867 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3bf03e-289c-4cdb-b2f9-38cf5cf7d6d5 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:39.611866 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 14/25] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 11:58:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:58:39.618280 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c2fe33-3a50-47ab-8922-01b9459f5818 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:39.618847 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c2fe33-3a50-47ab-8922-01b9459f5818 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:39.619525 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c2fe33-3a50-47ab-8922-01b9459f5818 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:39.663467 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:39.664415 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-94c2fe33-3a50-47ab-8922-01b9459f5818 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:58:39.665520 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-94c2fe33-3a50-47ab-8922-01b9459f5818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:39.720298 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 12/26] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 11:58:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:58:39.789291 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 15/27] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 11:58:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:58:39.966764 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfa1bb1-8886-4f19-b319-6b641646b8f9 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:40.090707 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfa1bb1-8886-4f19-b319-6b641646b8f9 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:40.128194 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c2fe33-3a50-47ab-8922-01b9459f5818 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:40.134232 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc3da7d-fe3a-46a4-9e59-8e2a2456554b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:40.134883 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc3da7d-fe3a-46a4-9e59-8e2a2456554b None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:40.135187 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc3da7d-fe3a-46a4-9e59-8e2a2456554b None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:40.135456 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc3da7d-fe3a-46a4-9e59-8e2a2456554b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:40.150349 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc3da7d-fe3a-46a4-9e59-8e2a2456554b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:58:40.156758 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-ebc3da7d-fe3a-46a4-9e59-8e2a2456554b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:58:40.156758 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:58:40.156758 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:58:40.156758 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:58:40.156758 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:58:40.156758 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:58:40.156758 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:58:40.156758 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:58:40.156758 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:58:40.156758 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:58:40.156758 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:58:40.156758 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:58:40.156758 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:58:40.156758 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:58:40.156758 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:58:40.156758 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:58:40.157600 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:58:40.157600 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:58:40.157600 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:58:40.157600 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:58:40.157600 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:58:40.157600 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:58:40.157600 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:58:40.157600 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:58:40.157600 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:58:40.157600 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:58:40.157600 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:58:40.157600 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 11:58:40.157600 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:58:40.157600 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:58:40.157600 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:58:40.157600 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:58:40.158350 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:58:40.158350 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:58:40.158350 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:58:40.158350 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:58:40.158350 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:58:40.158350 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:58:40.158350 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:58:40.158350 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:58:40.158350 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:58:40.158350 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:58:40.158350 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:58:40.158350 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 11:58:40.158350 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 13/28] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Sep 9 11:58:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:58:40.170929 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc3da7d-fe3a-46a4-9e59-8e2a2456554b None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:40.206634 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:40.272493 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:40.288426 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc3da7d-fe3a-46a4-9e59-8e2a2456554b None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:40.330462 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c2fe33-3a50-47ab-8922-01b9459f5818 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:40.357132 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9007035c-1e68-4216-b76b-92cd8258568b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:40.358038 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9007035c-1e68-4216-b76b-92cd8258568b None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:40.358459 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9007035c-1e68-4216-b76b-92cd8258568b None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:40.359845 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9007035c-1e68-4216-b76b-92cd8258568b None admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:40.360690 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9007035c-1e68-4216-b76b-92cd8258568b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:58:40.362479 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9007035c-1e68-4216-b76b-92cd8258568b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:40.366240 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9007035c-1e68-4216-b76b-92cd8258568b None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:40.403674 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eef976-111b-4412-9bb3-891976f987a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:40.404935 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eef976-111b-4412-9bb3-891976f987a0 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:40.405391 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eef976-111b-4412-9bb3-891976f987a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:40.405804 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eef976-111b-4412-9bb3-891976f987a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:40.405970 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 14/29] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 9 11:58:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:58:40.407243 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9007035c-1e68-4216-b76b-92cd8258568b None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:40.430749 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30eef976-111b-4412-9bb3-891976f987a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:40.434416 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30eef976-111b-4412-9bb3-891976f987a0 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:40.469049 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-30eef976-111b-4412-9bb3-891976f987a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:58:40.469049 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:58:40.469049 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:58:40.469049 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:58:40.469049 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:58:40.469049 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:58:40.469049 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:58:40.469049 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:58:40.469049 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:58:40.469049 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:58:40.469049 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:58:40.469049 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:58:40.469049 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:58:40.469049 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:58:40.469049 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:58:40.469049 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:58:40.470150 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:58:40.470150 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:58:40.470150 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:58:40.470150 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:58:40.470150 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:58:40.470150 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:58:40.470150 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:58:40.470150 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 11:58:40.470150 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:58:40.470150 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:58:40.470150 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:58:40.470150 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:58:40.470150 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:58:40.470150 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:58:40.470150 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:58:40.470150 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:58:40.470150 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:58:40.471152 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:58:40.471152 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:58:40.471152 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 11:58:40.476491 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 16/30] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 11:58:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:58:40.477461 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eef976-111b-4412-9bb3-891976f987a0 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:40.591287 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:40.595335 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9007035c-1e68-4216-b76b-92cd8258568b None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:40.636968 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cfccc6-f239-41d5-bb40-8821fd793c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:40.637642 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cfccc6-f239-41d5-bb40-8821fd793c43 None admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:40.637925 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cfccc6-f239-41d5-bb40-8821fd793c43 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:40.638197 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cfccc6-f239-41d5-bb40-8821fd793c43 None admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:40.638944 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cfccc6-f239-41d5-bb40-8821fd793c43 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:40.640339 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cfccc6-f239-41d5-bb40-8821fd793c43 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:40.645894 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eef976-111b-4412-9bb3-891976f987a0 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:40.674259 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:40.683086 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55d7641-c563-4d91-8e04-de9d16414d10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:40.683721 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55d7641-c563-4d91-8e04-de9d16414d10 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:40.684002 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55d7641-c563-4d91-8e04-de9d16414d10 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:40.684273 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55d7641-c563-4d91-8e04-de9d16414d10 None admin] PATH_INFO: `/v3/users/136011715ec34418adc0f536f6c7357c` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:40.685545 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55d7641-c563-4d91-8e04-de9d16414d10 None admin] RBAC: Authorizing `identity:get_user(user_id=136011715ec34418adc0f536f6c7357c)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:40.687089 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55d7641-c563-4d91-8e04-de9d16414d10 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:40.689351 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 17/31] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Sep 9 11:58:40 2022] GET /identity/v3/users/136011715ec34418adc0f536f6c7357c => generated 247 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:58:40.690314 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55d7641-c563-4d91-8e04-de9d16414d10 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:40.784630 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 15/32] 10.222.0.49 () {60 vars in 1131 bytes} [Fri Sep 9 11:58:40 2022] POST /identity/v3/roles => generated 219 bytes in 378 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:58:40.787258 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7697d95d-cc36-4766-8c36-413792d8b75f None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:40.788120 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7697d95d-cc36-4766-8c36-413792d8b75f None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:40.788612 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7697d95d-cc36-4766-8c36-413792d8b75f None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:40.790195 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 16/33] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 11:58:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:58:40.795881 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7697d95d-cc36-4766-8c36-413792d8b75f None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:40.798988 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55d7641-c563-4d91-8e04-de9d16414d10 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:40.837927 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea192aec-4421-4e27-83d5-88c4ff45491d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:40.838563 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea192aec-4421-4e27-83d5-88c4ff45491d None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:40.838844 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea192aec-4421-4e27-83d5-88c4ff45491d None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:40.839150 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea192aec-4421-4e27-83d5-88c4ff45491d None admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:40.839942 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea192aec-4421-4e27-83d5-88c4ff45491d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:58:40.840255 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea192aec-4421-4e27-83d5-88c4ff45491d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:40.842220 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea192aec-4421-4e27-83d5-88c4ff45491d None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:40.862811 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 18/34] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 9 11:58:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:58:40.864147 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70fc17b-c8ce-4754-9cfb-baf219ed987a None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:40.868927 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70fc17b-c8ce-4754-9cfb-baf219ed987a None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:40.868927 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70fc17b-c8ce-4754-9cfb-baf219ed987a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:40.884964 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-e70fc17b-c8ce-4754-9cfb-baf219ed987a None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:58:40.886324 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-e70fc17b-c8ce-4754-9cfb-baf219ed987a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:40.925743 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7697d95d-cc36-4766-8c36-413792d8b75f None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:40.942661 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:40.968436 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8390c454-1f85-4361-aa90-211e48cd0cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:40.969463 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8390c454-1f85-4361-aa90-211e48cd0cc4 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:40.969998 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8390c454-1f85-4361-aa90-211e48cd0cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:40.970428 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 19/35] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 11:58:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:58:40.970784 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8390c454-1f85-4361-aa90-211e48cd0cc4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:40.971306 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70fc17b-c8ce-4754-9cfb-baf219ed987a None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:40.980396 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8390c454-1f85-4361-aa90-211e48cd0cc4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:40.982516 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8390c454-1f85-4361-aa90-211e48cd0cc4 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:40.984657 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 17/36] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 11:58:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:58:40.991188 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8390c454-1f85-4361-aa90-211e48cd0cc4 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:41.038927 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:41.128782 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70fc17b-c8ce-4754-9cfb-baf219ed987a None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:41.172142 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcab8f9-c979-4b4a-be40-f82c7153e2d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:41.172787 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcab8f9-c979-4b4a-be40-f82c7153e2d4 None admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:41.173068 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcab8f9-c979-4b4a-be40-f82c7153e2d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:41.173341 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcab8f9-c979-4b4a-be40-f82c7153e2d4 None admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:41.174009 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcab8f9-c979-4b4a-be40-f82c7153e2d4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:41.175692 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcab8f9-c979-4b4a-be40-f82c7153e2d4 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:41.197192 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8390c454-1f85-4361-aa90-211e48cd0cc4 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:41.281083 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a082e81d-839b-4c43-b315-82b2000c1d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:41.281867 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a082e81d-839b-4c43-b315-82b2000c1d89 None admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:41.282225 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a082e81d-839b-4c43-b315-82b2000c1d89 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:41.282576 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a082e81d-839b-4c43-b315-82b2000c1d89 None admin] PATH_INFO: `/v3/domains/default/users/136011715ec34418adc0f536f6c7357c/roles/3f2ca33902a04b6dbc2ef7d06d7289d3` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:41.288116 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a082e81d-839b-4c43-b315-82b2000c1d89 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=136011715ec34418adc0f536f6c7357c, role_id=3f2ca33902a04b6dbc2ef7d06d7289d3)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:41.289721 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a082e81d-839b-4c43-b315-82b2000c1d89 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:41.312518 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 18/37] 10.222.0.49 () {58 vars in 1361 bytes} [Fri Sep 9 11:58:40 2022] PUT /identity/v3/domains/default/users/136011715ec34418adc0f536f6c7357c/roles/3f2ca33902a04b6dbc2ef7d06d7289d3 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:58:41.314161 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59aa41a-e98b-4654-b361-0d8195568147 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:41.314454 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59aa41a-e98b-4654-b361-0d8195568147 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:41.314788 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59aa41a-e98b-4654-b361-0d8195568147 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:41.316241 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 19/38] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 11:58:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:58:41.317192 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38675ac8-78a7-4151-a42e-0b7691346053 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:41.317535 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38675ac8-78a7-4151-a42e-0b7691346053 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:41.318347 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38675ac8-78a7-4151-a42e-0b7691346053 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:41.319698 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 20/39] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 11:58:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:58:41.325462 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ff6c95-9ee7-4791-887e-15e428d6d0d8 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:41.325823 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ff6c95-9ee7-4791-887e-15e428d6d0d8 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:41.326173 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ff6c95-9ee7-4791-887e-15e428d6d0d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:41.337518 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 20/40] 10.222.0.49 () {60 vars in 1141 bytes} [Fri Sep 9 11:58:40 2022] POST /identity/v3/projects => generated 301 bytes in 367 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:58:41.339356 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e5418c-394e-4464-af3e-4ec207a0168f None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:41.339811 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e5418c-394e-4464-af3e-4ec207a0168f None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:41.340227 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e5418c-394e-4464-af3e-4ec207a0168f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:41.347474 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-e7ff6c95-9ee7-4791-887e-15e428d6d0d8 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:58:41.351133 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-e7ff6c95-9ee7-4791-887e-15e428d6d0d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:41.361587 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-75e5418c-394e-4464-af3e-4ec207a0168f None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:58:41.363645 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-75e5418c-394e-4464-af3e-4ec207a0168f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:41.399247 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:41.467629 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 21/41] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 11:58:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:58:41.475721 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e5418c-394e-4464-af3e-4ec207a0168f None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:41.489056 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 21/42] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 11:58:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:58:41.605193 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e5418c-394e-4464-af3e-4ec207a0168f None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:41.753412 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6175015e-cab3-4f3a-8a9e-eecaddd8abc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:41.754042 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6175015e-cab3-4f3a-8a9e-eecaddd8abc7 None admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:41.754376 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6175015e-cab3-4f3a-8a9e-eecaddd8abc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:41.754657 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6175015e-cab3-4f3a-8a9e-eecaddd8abc7 None admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:41.755418 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6175015e-cab3-4f3a-8a9e-eecaddd8abc7 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:41.756958 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6175015e-cab3-4f3a-8a9e-eecaddd8abc7 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:41.780740 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 22/43] 10.222.0.49 () {60 vars in 1131 bytes} [Fri Sep 9 11:58:41 2022] POST /identity/v3/roles => generated 225 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:58:41.796154 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ff6c95-9ee7-4791-887e-15e428d6d0d8 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:41.821761 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:41.836509 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6175015e-cab3-4f3a-8a9e-eecaddd8abc7 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:41.881936 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:41.950461 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ff6c95-9ee7-4791-887e-15e428d6d0d8 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:41.978690 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6175015e-cab3-4f3a-8a9e-eecaddd8abc7 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:41.995852 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521d848c-a965-4344-ba76-2fd95fd2c875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:41.996415 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521d848c-a965-4344-ba76-2fd95fd2c875 None admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:41.996694 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521d848c-a965-4344-ba76-2fd95fd2c875 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:41.996966 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521d848c-a965-4344-ba76-2fd95fd2c875 None admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:41.997646 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521d848c-a965-4344-ba76-2fd95fd2c875 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:41.999055 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521d848c-a965-4344-ba76-2fd95fd2c875 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:42.016993 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 22/44] 10.222.0.49 () {60 vars in 1131 bytes} [Fri Sep 9 11:58:41 2022] POST /identity/v3/roles => generated 223 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 11:58:42.018500 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80adbe8-8f18-455c-9f34-2bcb6009d8a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:42.019209 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80adbe8-8f18-455c-9f34-2bcb6009d8a3 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:42.019494 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80adbe8-8f18-455c-9f34-2bcb6009d8a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:42.019767 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80adbe8-8f18-455c-9f34-2bcb6009d8a3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:42.021467 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80adbe8-8f18-455c-9f34-2bcb6009d8a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:42.024196 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80adbe8-8f18-455c-9f34-2bcb6009d8a3 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:42.025069 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 23/45] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 11:58:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:58:42.031954 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521d848c-a965-4344-ba76-2fd95fd2c875 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:42.127637 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:42.155333 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521d848c-a965-4344-ba76-2fd95fd2c875 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:42.193483 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1295252-f84b-468c-b579-fe08d36aa681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:42.194038 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1295252-f84b-468c-b579-fe08d36aa681 None admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:42.194322 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1295252-f84b-468c-b579-fe08d36aa681 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:42.194601 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1295252-f84b-468c-b579-fe08d36aa681 None admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:42.195287 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1295252-f84b-468c-b579-fe08d36aa681 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:42.196755 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1295252-f84b-468c-b579-fe08d36aa681 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:42.227839 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 23/46] 10.222.0.49 () {60 vars in 1141 bytes} [Fri Sep 9 11:58:42 2022] POST /identity/v3/projects => generated 312 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:58:43.637892 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3cbc8a-462b-48cb-8b1a-d5467975ec1c None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:43.638400 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3cbc8a-462b-48cb-8b1a-d5467975ec1c None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:43.638516 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3cbc8a-462b-48cb-8b1a-d5467975ec1c None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:43.639826 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 24/47] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 11:58:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:58:43.645168 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17c38a6-7ede-48c3-907c-3e1b36b155c2 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:43.645514 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17c38a6-7ede-48c3-907c-3e1b36b155c2 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:43.645617 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17c38a6-7ede-48c3-907c-3e1b36b155c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:43.665080 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-c17c38a6-7ede-48c3-907c-3e1b36b155c2 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:58:43.666248 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-c17c38a6-7ede-48c3-907c-3e1b36b155c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:43.731582 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 24/48] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 11:58:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:58:44.001834 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3cbc8a-462b-48cb-8b1a-d5467975ec1c None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:44.098319 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:44.107971 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3cbc8a-462b-48cb-8b1a-d5467975ec1c None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:44.143139 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9865bb31-a9bb-462a-8737-e709b637db95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:44.143777 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9865bb31-a9bb-462a-8737-e709b637db95 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:44.144060 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9865bb31-a9bb-462a-8737-e709b637db95 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:44.144334 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9865bb31-a9bb-462a-8737-e709b637db95 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:44.154975 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9865bb31-a9bb-462a-8737-e709b637db95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:58:44.159822 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-9865bb31-a9bb-462a-8737-e709b637db95 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:58:44.159822 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:58:44.159822 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:58:44.159822 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:58:44.159822 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:58:44.159822 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:58:44.159822 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:58:44.159822 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:58:44.159822 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:58:44.159822 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:58:44.159822 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:58:44.159822 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:58:44.159822 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:58:44.159822 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:58:44.159822 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:58:44.159822 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:58:44.160745 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:58:44.160745 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:58:44.160745 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:58:44.160745 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:58:44.160745 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:58:44.160745 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:58:44.160745 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:58:44.160745 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:58:44.160745 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:58:44.160745 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:58:44.160745 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:58:44.160745 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 11:58:44.160745 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:58:44.160745 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:58:44.160745 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:58:44.160745 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:58:44.162419 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:58:44.162419 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:58:44.162419 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:58:44.162419 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:58:44.162419 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:58:44.162419 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:58:44.162419 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:58:44.162419 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:58:44.162419 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:58:44.162419 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:58:44.162419 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:58:44.162419 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 11:58:44.162419 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 25/49] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Sep 9 11:58:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:58:44.167581 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17c38a6-7ede-48c3-907c-3e1b36b155c2 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:44.230271 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:44.272291 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17c38a6-7ede-48c3-907c-3e1b36b155c2 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:44.308743 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d51a2a-79a5-4151-80b1-75b20ef36628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:44.309390 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d51a2a-79a5-4151-80b1-75b20ef36628 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:44.309673 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d51a2a-79a5-4151-80b1-75b20ef36628 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:44.309945 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d51a2a-79a5-4151-80b1-75b20ef36628 None admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:44.310613 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d51a2a-79a5-4151-80b1-75b20ef36628 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:58:44.310939 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d51a2a-79a5-4151-80b1-75b20ef36628 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:44.312359 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d51a2a-79a5-4151-80b1-75b20ef36628 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:44.321323 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 25/50] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 9 11:58:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:58:44.327475 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9865bb31-a9bb-462a-8737-e709b637db95 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:44.392491 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b45e48d-3f40-4db2-bd23-2afdf82e7ff4 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:44.392785 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b45e48d-3f40-4db2-bd23-2afdf82e7ff4 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:44.393061 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b45e48d-3f40-4db2-bd23-2afdf82e7ff4 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:44.394416 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 26/51] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 11:58:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 11:58:44.400064 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0471d9d2-c95a-4037-af83-a528e193d7da None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:44.400351 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0471d9d2-c95a-4037-af83-a528e193d7da None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:44.400628 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0471d9d2-c95a-4037-af83-a528e193d7da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:44.419110 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-0471d9d2-c95a-4037-af83-a528e193d7da None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:58:44.420318 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-0471d9d2-c95a-4037-af83-a528e193d7da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:44.426014 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9865bb31-a9bb-462a-8737-e709b637db95 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:44.461460 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b7a33f-4b50-4acd-bba5-37352da1bc76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:44.462096 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b7a33f-4b50-4acd-bba5-37352da1bc76 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:44.462380 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b7a33f-4b50-4acd-bba5-37352da1bc76 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:44.462657 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b7a33f-4b50-4acd-bba5-37352da1bc76 None admin] PATH_INFO: `/v3/users/136011715ec34418adc0f536f6c7357c` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:44.463876 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b7a33f-4b50-4acd-bba5-37352da1bc76 None admin] RBAC: Authorizing `identity:get_user(user_id=136011715ec34418adc0f536f6c7357c)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:44.465374 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b7a33f-4b50-4acd-bba5-37352da1bc76 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:44.467076 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 26/52] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Sep 9 11:58:44 2022] GET /identity/v3/users/136011715ec34418adc0f536f6c7357c => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:58:44.472728 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b7a33f-4b50-4acd-bba5-37352da1bc76 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:44.499907 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:44.583371 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b7a33f-4b50-4acd-bba5-37352da1bc76 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:44.595059 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 27/53] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 11:58:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 11:58:44.617864 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea728b4a-1b6f-4549-99a9-aa99350a2c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:44.618493 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea728b4a-1b6f-4549-99a9-aa99350a2c39 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:44.618774 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea728b4a-1b6f-4549-99a9-aa99350a2c39 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:44.619073 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea728b4a-1b6f-4549-99a9-aa99350a2c39 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:44.620191 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea728b4a-1b6f-4549-99a9-aa99350a2c39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:44.621648 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea728b4a-1b6f-4549-99a9-aa99350a2c39 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:44.623247 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 27/54] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 11:58:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:58:44.629076 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0471d9d2-c95a-4037-af83-a528e193d7da None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:44.680521 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:44.731355 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0471d9d2-c95a-4037-af83-a528e193d7da None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:44.765077 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec27932-8fa0-49b1-a003-034c507139cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:44.765682 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec27932-8fa0-49b1-a003-034c507139cf None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:44.765961 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec27932-8fa0-49b1-a003-034c507139cf None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:44.766237 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec27932-8fa0-49b1-a003-034c507139cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:44.766989 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec27932-8fa0-49b1-a003-034c507139cf None admin] RBAC: Adding query filter params (role.id=3f2ca33902a04b6dbc2ef7d06d7289d3, scope.domain.id=default, user.id=136011715ec34418adc0f536f6c7357c) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:58:44.767299 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec27932-8fa0-49b1-a003-034c507139cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:44.768742 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec27932-8fa0-49b1-a003-034c507139cf None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:44.777008 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 28/55] 10.222.0.49 () {56 vars in 1322 bytes} [Fri Sep 9 11:58:44 2022] GET /identity/v3/role_assignments?user.id=136011715ec34418adc0f536f6c7357c&scope.domain.id=default&role.id=3f2ca33902a04b6dbc2ef7d06d7289d3 => generated 530 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:58:44.866055 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea728b4a-1b6f-4549-99a9-aa99350a2c39 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:44.959701 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:44.970253 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea728b4a-1b6f-4549-99a9-aa99350a2c39 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:45.005626 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e64e91a-7871-448f-a3ec-411913ba898f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:45.006407 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e64e91a-7871-448f-a3ec-411913ba898f None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:45.006692 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e64e91a-7871-448f-a3ec-411913ba898f None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:45.006998 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e64e91a-7871-448f-a3ec-411913ba898f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:45.008120 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e64e91a-7871-448f-a3ec-411913ba898f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:45.009599 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e64e91a-7871-448f-a3ec-411913ba898f None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:45.011196 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 28/56] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 11:58:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:58:45.016878 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec27932-8fa0-49b1-a003-034c507139cf None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:45.114132 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec27932-8fa0-49b1-a003-034c507139cf None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:45.127435 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:45.147861 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33f3a57-d28f-4145-82fe-2036494d8be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:45.148458 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33f3a57-d28f-4145-82fe-2036494d8be7 None admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:45.148736 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33f3a57-d28f-4145-82fe-2036494d8be7 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:45.149009 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33f3a57-d28f-4145-82fe-2036494d8be7 None admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:45.149647 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33f3a57-d28f-4145-82fe-2036494d8be7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:45.151121 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33f3a57-d28f-4145-82fe-2036494d8be7 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:45.175411 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 29/57] 10.222.0.49 () {60 vars in 1140 bytes} [Fri Sep 9 11:58:45 2022] POST /identity/v3/projects => generated 298 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:58:47.347120 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7490bbcd-ef9e-4736-86e8-a42962614263 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:47.347638 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7490bbcd-ef9e-4736-86e8-a42962614263 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:47.347741 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7490bbcd-ef9e-4736-86e8-a42962614263 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:47.349007 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 29/58] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 11:58:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:58:47.355087 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a829c92-d014-4793-ad88-6c5f97ec67bd None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:47.355502 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a829c92-d014-4793-ad88-6c5f97ec67bd None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:47.355635 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a829c92-d014-4793-ad88-6c5f97ec67bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:47.374395 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-6a829c92-d014-4793-ad88-6c5f97ec67bd None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:58:47.375591 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-6a829c92-d014-4793-ad88-6c5f97ec67bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:47.440933 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 30/59] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 11:58:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:58:47.718850 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7490bbcd-ef9e-4736-86e8-a42962614263 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:47.816815 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7490bbcd-ef9e-4736-86e8-a42962614263 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:47.849256 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fc863d-668d-4484-b3e5-640fe0ac8c4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:47.849968 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fc863d-668d-4484-b3e5-640fe0ac8c4c None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:47.850246 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fc863d-668d-4484-b3e5-640fe0ac8c4c None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:47.850516 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fc863d-668d-4484-b3e5-640fe0ac8c4c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:47.851640 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fc863d-668d-4484-b3e5-640fe0ac8c4c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:47.853073 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fc863d-668d-4484-b3e5-640fe0ac8c4c None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:47.854630 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 30/60] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 11:58:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:58:47.860094 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a829c92-d014-4793-ad88-6c5f97ec67bd None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:47.950646 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:47.956762 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a829c92-d014-4793-ad88-6c5f97ec67bd None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:47.991030 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2165df22-2819-416a-9fb3-15c110f64d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:47.991634 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2165df22-2819-416a-9fb3-15c110f64d76 None admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:47.991914 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2165df22-2819-416a-9fb3-15c110f64d76 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:47.992184 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2165df22-2819-416a-9fb3-15c110f64d76 None admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:47.992826 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2165df22-2819-416a-9fb3-15c110f64d76 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:47.994260 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2165df22-2819-416a-9fb3-15c110f64d76 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:48.020218 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 31/61] 10.222.0.49 () {60 vars in 1132 bytes} [Fri Sep 9 11:58:47 2022] POST /identity/v3/users => generated 275 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:58:51.479819 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f1b8a1-b008-4c3e-97b6-f19bb3f22a0f None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:51.480331 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f1b8a1-b008-4c3e-97b6-f19bb3f22a0f None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:51.480405 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f1b8a1-b008-4c3e-97b6-f19bb3f22a0f None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:51.481933 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 31/62] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 11:58:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:58:51.487156 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3d92f5-7247-4615-b28b-ee844adae317 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:51.487956 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3d92f5-7247-4615-b28b-ee844adae317 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:51.488428 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3d92f5-7247-4615-b28b-ee844adae317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:51.516144 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-ef3d92f5-7247-4615-b28b-ee844adae317 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:58:51.517368 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-ef3d92f5-7247-4615-b28b-ee844adae317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:51.517544 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938058a0-11a2-47e0-bc5c-d44c7dc75a5d None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:51.517816 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938058a0-11a2-47e0-bc5c-d44c7dc75a5d None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:51.518095 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938058a0-11a2-47e0-bc5c-d44c7dc75a5d None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:51.519939 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 32/63] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 11:58:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:58:51.537608 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f19eb9-92a2-43c4-bcd8-0787126c4c60 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:51.538261 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f19eb9-92a2-43c4-bcd8-0787126c4c60 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:51.538930 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f19eb9-92a2-43c4-bcd8-0787126c4c60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:51.569532 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-26f19eb9-92a2-43c4-bcd8-0787126c4c60 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:58:51.575217 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-26f19eb9-92a2-43c4-bcd8-0787126c4c60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:51.607254 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:51.627164 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 32/64] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 11:58:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:58:51.667114 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 33/65] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 11:58:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 11:58:51.758793 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189aebf6-05a0-4b51-bd15-46b293be8a86 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:51.759148 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189aebf6-05a0-4b51-bd15-46b293be8a86 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:51.759377 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189aebf6-05a0-4b51-bd15-46b293be8a86 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:51.760869 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 33/66] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 11:58:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:58:51.766143 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcefaea-aff3-4a13-ab87-b00b3b003299 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:51.766438 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcefaea-aff3-4a13-ab87-b00b3b003299 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:51.766715 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcefaea-aff3-4a13-ab87-b00b3b003299 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:51.782729 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff35162-1353-48b5-a356-c3bd080bc736 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:51.783058 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff35162-1353-48b5-a356-c3bd080bc736 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:51.783307 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff35162-1353-48b5-a356-c3bd080bc736 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:51.784972 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 34/67] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 11:58:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:58:51.789671 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8a1a68-a2ca-42c2-87ef-ed50ac230379 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:51.790162 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8a1a68-a2ca-42c2-87ef-ed50ac230379 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:51.790638 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8a1a68-a2ca-42c2-87ef-ed50ac230379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:51.799406 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-8bcefaea-aff3-4a13-ab87-b00b3b003299 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:58:51.803201 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-8bcefaea-aff3-4a13-ab87-b00b3b003299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:51.820005 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-6c8a1a68-a2ca-42c2-87ef-ed50ac230379 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:58:51.821590 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-6c8a1a68-a2ca-42c2-87ef-ed50ac230379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:51.935418 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 34/68] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 11:58:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:58:51.945358 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 35/69] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 11:58:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:58:51.949927 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5035758b-e69a-4d51-8a4b-9391e46b2575 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:51.950226 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5035758b-e69a-4d51-8a4b-9391e46b2575 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:51.950505 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5035758b-e69a-4d51-8a4b-9391e46b2575 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:51.952097 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 35/70] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 11:58:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:58:51.964369 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920dae39-88fe-4d6d-a9fd-a9fd4b98441e None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:51.964659 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920dae39-88fe-4d6d-a9fd-a9fd4b98441e None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:51.964937 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920dae39-88fe-4d6d-a9fd-a9fd4b98441e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:51.971075 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5035758b-e69a-4d51-8a4b-9391e46b2575 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:51.993345 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-920dae39-88fe-4d6d-a9fd-a9fd4b98441e None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:58:51.994781 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-920dae39-88fe-4d6d-a9fd-a9fd4b98441e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:52.004613 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:52.067284 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 36/71] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 11:58:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Sep 09 11:58:52.069258 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920dae39-88fe-4d6d-a9fd-a9fd4b98441e None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:52.095272 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:52.109620 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5035758b-e69a-4d51-8a4b-9391e46b2575 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:52.154556 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6b9602-b48c-4674-9bc0-e3419e1ffd82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:52.155252 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6b9602-b48c-4674-9bc0-e3419e1ffd82 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:52.155539 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6b9602-b48c-4674-9bc0-e3419e1ffd82 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:52.155815 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6b9602-b48c-4674-9bc0-e3419e1ffd82 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:52.167918 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6b9602-b48c-4674-9bc0-e3419e1ffd82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:58:52.174382 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-ac6b9602-b48c-4674-9bc0-e3419e1ffd82 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 11:58:52.174382 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:58:52.174382 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:58:52.174382 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:58:52.174382 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:58:52.174382 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:58:52.174382 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:58:52.174382 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:58:52.174382 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:58:52.174382 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:58:52.174382 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:58:52.174382 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:58:52.174382 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:58:52.174382 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:58:52.174382 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:58:52.174382 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:58:52.175303 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:58:52.175303 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:58:52.175303 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:58:52.175303 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:58:52.175303 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:58:52.175303 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:58:52.175303 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:58:52.175303 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:58:52.175303 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:58:52.175303 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:58:52.175303 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:58:52.175303 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 11:58:52.175303 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:58:52.175303 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:58:52.175303 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:58:52.175303 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:58:52.176045 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:58:52.176045 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:58:52.176045 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:58:52.176045 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:58:52.176045 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:58:52.176045 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:58:52.176045 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:58:52.176045 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:58:52.176045 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:58:52.176045 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:58:52.176045 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 11:58:52.176045 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 11:58:52.179119 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 36/72] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 9 11:58:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:58:52.187010 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6b9602-b48c-4674-9bc0-e3419e1ffd82 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:52.204990 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920dae39-88fe-4d6d-a9fd-a9fd4b98441e None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:52.257467 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46ee252-7fdb-4034-85ea-842dfb1f3a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:52.258113 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46ee252-7fdb-4034-85ea-842dfb1f3a88 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:52.258398 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46ee252-7fdb-4034-85ea-842dfb1f3a88 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:52.258676 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46ee252-7fdb-4034-85ea-842dfb1f3a88 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:52.270829 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46ee252-7fdb-4034-85ea-842dfb1f3a88 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:58:52.277518 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-d46ee252-7fdb-4034-85ea-842dfb1f3a88 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:58:52.277518 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:58:52.277518 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:58:52.277518 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:58:52.277518 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:58:52.277518 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:58:52.277518 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:58:52.277518 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:58:52.277518 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:58:52.277518 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:58:52.277518 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:58:52.277518 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:58:52.277518 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:58:52.277518 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:58:52.277518 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:58:52.277518 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:58:52.278572 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:58:52.278572 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:58:52.278572 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:58:52.278572 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:58:52.278572 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:58:52.278572 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:58:52.278572 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:58:52.278572 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:58:52.278572 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:58:52.278572 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:58:52.278572 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:58:52.278572 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 11:58:52.278572 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:58:52.278572 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:58:52.278572 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:58:52.278572 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:58:52.279543 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:58:52.279543 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:58:52.279543 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:58:52.279543 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:58:52.279543 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:58:52.279543 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:58:52.279543 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:58:52.279543 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:58:52.279543 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:58:52.279543 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:58:52.279543 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:58:52.279543 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 11:58:52.281610 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 37/73] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Sep 9 11:58:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:58:52.283211 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46ee252-7fdb-4034-85ea-842dfb1f3a88 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:52.287300 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:52.324024 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6b9602-b48c-4674-9bc0-e3419e1ffd82 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:52.361090 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c701859-5ccd-4c27-adcf-d51e4e06f0c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:52.361691 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c701859-5ccd-4c27-adcf-d51e4e06f0c7 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:52.361969 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c701859-5ccd-4c27-adcf-d51e4e06f0c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:52.362237 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c701859-5ccd-4c27-adcf-d51e4e06f0c7 None admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:52.362893 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c701859-5ccd-4c27-adcf-d51e4e06f0c7 None admin] RBAC: Adding query filter params (name=member) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:58:52.363351 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c701859-5ccd-4c27-adcf-d51e4e06f0c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:52.364812 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c701859-5ccd-4c27-adcf-d51e4e06f0c7 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:52.373681 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 37/74] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Sep 9 11:58:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:58:52.374590 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c701859-5ccd-4c27-adcf-d51e4e06f0c7 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:52.387378 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:52.404134 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46ee252-7fdb-4034-85ea-842dfb1f3a88 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:52.439407 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c49a9b-168f-46ce-895a-e2fc3c239b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:52.440040 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c49a9b-168f-46ce-895a-e2fc3c239b1c None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:52.440317 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c49a9b-168f-46ce-895a-e2fc3c239b1c None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:52.440599 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c49a9b-168f-46ce-895a-e2fc3c239b1c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:52.441722 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c49a9b-168f-46ce-895a-e2fc3c239b1c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:52.443259 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c49a9b-168f-46ce-895a-e2fc3c239b1c None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:52.444861 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 38/75] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 11:58:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:58:52.445624 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c49a9b-168f-46ce-895a-e2fc3c239b1c None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:52.482768 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c701859-5ccd-4c27-adcf-d51e4e06f0c7 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:52.514381 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:52.519544 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6777afc1-d74e-44f8-8cab-775e7a089d81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:52.520165 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6777afc1-d74e-44f8-8cab-775e7a089d81 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:52.520449 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6777afc1-d74e-44f8-8cab-775e7a089d81 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:52.520726 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6777afc1-d74e-44f8-8cab-775e7a089d81 None admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:52.522067 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6777afc1-d74e-44f8-8cab-775e7a089d81 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:58:52.522378 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6777afc1-d74e-44f8-8cab-775e7a089d81 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:52.523815 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6777afc1-d74e-44f8-8cab-775e7a089d81 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:52.531921 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 38/76] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 9 11:58:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:58:52.532713 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6777afc1-d74e-44f8-8cab-775e7a089d81 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:52.549180 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c49a9b-168f-46ce-895a-e2fc3c239b1c None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:52.585503 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9d12ed-3245-4864-8ec8-edff7e9ad527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:52.586061 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9d12ed-3245-4864-8ec8-edff7e9ad527 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:52.586343 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9d12ed-3245-4864-8ec8-edff7e9ad527 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:52.586612 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9d12ed-3245-4864-8ec8-edff7e9ad527 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:52.596483 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9d12ed-3245-4864-8ec8-edff7e9ad527 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:58:52.601991 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-6b9d12ed-3245-4864-8ec8-edff7e9ad527 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 11:58:52.601991 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:58:52.601991 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:58:52.601991 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:58:52.601991 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:58:52.601991 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:58:52.601991 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:58:52.601991 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:58:52.601991 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:58:52.601991 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:58:52.601991 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:58:52.601991 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:58:52.601991 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:58:52.601991 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:58:52.601991 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:58:52.601991 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:58:52.603043 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:58:52.603043 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:58:52.603043 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:58:52.603043 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:58:52.603043 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:58:52.603043 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:58:52.603043 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:58:52.603043 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:58:52.603043 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:58:52.603043 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:58:52.603043 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:58:52.603043 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 11:58:52.603043 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:58:52.603043 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:58:52.603043 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:58:52.603043 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:58:52.607861 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:58:52.607861 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:58:52.607861 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:58:52.607861 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:58:52.607861 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:58:52.607861 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:58:52.607861 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:58:52.607861 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:58:52.607861 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:58:52.607861 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:58:52.607861 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 11:58:52.607861 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 11:58:52.607861 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 39/77] 10.222.0.49 () {56 vars in 1114 bytes} [Fri Sep 9 11:58:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:58:52.607861 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9d12ed-3245-4864-8ec8-edff7e9ad527 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:52.610159 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:52.642503 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6777afc1-d74e-44f8-8cab-775e7a089d81 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:52.675928 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d514ed2-9676-4a92-b329-4fb7bd792b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:52.676561 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d514ed2-9676-4a92-b329-4fb7bd792b57 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:52.676841 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d514ed2-9676-4a92-b329-4fb7bd792b57 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:52.677121 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d514ed2-9676-4a92-b329-4fb7bd792b57 None admin] PATH_INFO: `/v3/users/1ccfba61dc7d4ad98da496e5800a8589` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:52.691204 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d514ed2-9676-4a92-b329-4fb7bd792b57 None admin] RBAC: Authorizing `identity:get_user(user_id=1ccfba61dc7d4ad98da496e5800a8589)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:52.692654 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d514ed2-9676-4a92-b329-4fb7bd792b57 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:52.694310 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 39/78] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Sep 9 11:58:52 2022] GET /identity/v3/users/1ccfba61dc7d4ad98da496e5800a8589 => generated 275 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:58:52.695079 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d514ed2-9676-4a92-b329-4fb7bd792b57 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:52.708223 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9d12ed-3245-4864-8ec8-edff7e9ad527 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:52.739251 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:52.746430 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e23b29e-c4cb-4e53-9085-39f395ed6660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:52.747091 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e23b29e-c4cb-4e53-9085-39f395ed6660 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:52.747372 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e23b29e-c4cb-4e53-9085-39f395ed6660 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:52.747644 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e23b29e-c4cb-4e53-9085-39f395ed6660 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:52.757701 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e23b29e-c4cb-4e53-9085-39f395ed6660 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:58:52.762496 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-5e23b29e-c4cb-4e53-9085-39f395ed6660 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 11:58:52.762496 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:58:52.762496 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:58:52.762496 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:58:52.762496 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:58:52.762496 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:58:52.762496 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:58:52.762496 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:58:52.762496 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:58:52.762496 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:58:52.762496 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:58:52.762496 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:58:52.762496 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:58:52.762496 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:58:52.762496 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:58:52.762496 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:58:52.763381 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:58:52.763381 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:58:52.763381 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:58:52.763381 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:58:52.763381 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:58:52.763381 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:58:52.763381 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:58:52.763381 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:58:52.763381 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:58:52.763381 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:58:52.763381 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:58:52.763381 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 11:58:52.763381 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:58:52.763381 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:58:52.763381 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:58:52.763381 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:58:52.764175 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:58:52.764175 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:58:52.764175 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:58:52.764175 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:58:52.764175 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:58:52.764175 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:58:52.764175 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:58:52.764175 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:58:52.764175 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:58:52.764175 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:58:52.764175 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 11:58:52.764175 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 11:58:52.764175 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 40/79] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 9 11:58:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:58:52.764847 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e23b29e-c4cb-4e53-9085-39f395ed6660 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:52.797915 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d514ed2-9676-4a92-b329-4fb7bd792b57 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:52.824875 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:52.833459 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3881a530-ee15-4e5c-aab5-0024997b14e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:52.834712 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3881a530-ee15-4e5c-aab5-0024997b14e9 None admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:52.835032 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3881a530-ee15-4e5c-aab5-0024997b14e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:52.835311 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3881a530-ee15-4e5c-aab5-0024997b14e9 None admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:52.836023 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3881a530-ee15-4e5c-aab5-0024997b14e9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:52.837421 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3881a530-ee15-4e5c-aab5-0024997b14e9 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:52.863586 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 40/80] 10.222.0.49 () {60 vars in 1132 bytes} [Fri Sep 9 11:58:52 2022] POST /identity/v3/users => generated 289 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:58:52.864402 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3881a530-ee15-4e5c-aab5-0024997b14e9 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:52.865354 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e23b29e-c4cb-4e53-9085-39f395ed6660 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:52.913072 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b77663-8a07-4e5c-aede-7c71d45ce18a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:52.913756 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b77663-8a07-4e5c-aede-7c71d45ce18a None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:52.914040 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b77663-8a07-4e5c-aede-7c71d45ce18a None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:52.914318 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b77663-8a07-4e5c-aede-7c71d45ce18a None admin] PATH_INFO: `/v3/users/136011715ec34418adc0f536f6c7357c` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:52.915549 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b77663-8a07-4e5c-aede-7c71d45ce18a None admin] RBAC: Authorizing `identity:get_user(user_id=136011715ec34418adc0f536f6c7357c)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:52.917059 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b77663-8a07-4e5c-aede-7c71d45ce18a None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:52.918977 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 41/81] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Sep 9 11:58:52 2022] GET /identity/v3/users/136011715ec34418adc0f536f6c7357c => generated 247 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:58:52.924581 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b77663-8a07-4e5c-aede-7c71d45ce18a None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:52.985631 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3881a530-ee15-4e5c-aab5-0024997b14e9 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:52.992937 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:53.023274 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfec5bc0-402c-45a0-96cc-2cb3b3f9bc19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:53.023930 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfec5bc0-402c-45a0-96cc-2cb3b3f9bc19 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:53.024193 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfec5bc0-402c-45a0-96cc-2cb3b3f9bc19 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:53.024473 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfec5bc0-402c-45a0-96cc-2cb3b3f9bc19 None admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:53.025168 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfec5bc0-402c-45a0-96cc-2cb3b3f9bc19 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:58:53.025474 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfec5bc0-402c-45a0-96cc-2cb3b3f9bc19 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:53.026934 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfec5bc0-402c-45a0-96cc-2cb3b3f9bc19 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:53.035423 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 41/82] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 9 11:58:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:58:53.041176 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfec5bc0-402c-45a0-96cc-2cb3b3f9bc19 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:53.043407 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b77663-8a07-4e5c-aede-7c71d45ce18a None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:53.088404 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95068b97-26d2-42a4-9f32-7c266a1252df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:53.089239 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95068b97-26d2-42a4-9f32-7c266a1252df None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:53.089678 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95068b97-26d2-42a4-9f32-7c266a1252df None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:53.090091 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95068b97-26d2-42a4-9f32-7c266a1252df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:53.101184 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:53.165614 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfec5bc0-402c-45a0-96cc-2cb3b3f9bc19 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:53.171900 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95068b97-26d2-42a4-9f32-7c266a1252df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:53.173467 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95068b97-26d2-42a4-9f32-7c266a1252df None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:53.206777 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0233f9dc-225a-4db5-b8ac-4a90c381b343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:53.207429 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0233f9dc-225a-4db5-b8ac-4a90c381b343 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:53.207714 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0233f9dc-225a-4db5-b8ac-4a90c381b343 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:53.207994 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0233f9dc-225a-4db5-b8ac-4a90c381b343 None admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:53.209200 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0233f9dc-225a-4db5-b8ac-4a90c381b343 None admin] RBAC: Adding query filter params (name=member) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:58:53.209508 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0233f9dc-225a-4db5-b8ac-4a90c381b343 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:53.210938 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0233f9dc-225a-4db5-b8ac-4a90c381b343 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:53.222630 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 42/83] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Sep 9 11:58:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:58:53.223614 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0233f9dc-225a-4db5-b8ac-4a90c381b343 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:53.239635 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:53.298138 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 42/84] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 9 11:58:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 11:58:53.299127 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95068b97-26d2-42a4-9f32-7c266a1252df None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:53.369031 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0233f9dc-225a-4db5-b8ac-4a90c381b343 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:53.379044 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:53.407962 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95068b97-26d2-42a4-9f32-7c266a1252df None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:53.408748 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2183d8-cebd-4505-a45f-f7ccd94d0a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:53.409702 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2183d8-cebd-4505-a45f-f7ccd94d0a5f None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:53.410210 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2183d8-cebd-4505-a45f-f7ccd94d0a5f None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:53.410621 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2183d8-cebd-4505-a45f-f7ccd94d0a5f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:53.447220 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27783c03-cc11-4feb-b13c-27a2b8742871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:53.447918 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27783c03-cc11-4feb-b13c-27a2b8742871 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:53.448203 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27783c03-cc11-4feb-b13c-27a2b8742871 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:53.448480 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27783c03-cc11-4feb-b13c-27a2b8742871 None admin] PATH_INFO: `/v3/users/1ccfba61dc7d4ad98da496e5800a8589` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:53.449988 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27783c03-cc11-4feb-b13c-27a2b8742871 None admin] RBAC: Authorizing `identity:get_user(user_id=1ccfba61dc7d4ad98da496e5800a8589)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:53.451507 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27783c03-cc11-4feb-b13c-27a2b8742871 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:53.453331 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 43/85] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Sep 9 11:58:53 2022] GET /identity/v3/users/1ccfba61dc7d4ad98da496e5800a8589 => generated 275 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:58:53.454095 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27783c03-cc11-4feb-b13c-27a2b8742871 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:53.494651 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:53.500721 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2183d8-cebd-4505-a45f-f7ccd94d0a5f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:53.502342 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2183d8-cebd-4505-a45f-f7ccd94d0a5f None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:53.583987 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27783c03-cc11-4feb-b13c-27a2b8742871 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:53.618492 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 43/86] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 9 11:58:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 11:58:53.623926 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2183d8-cebd-4505-a45f-f7ccd94d0a5f None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:53.629704 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b601dd2-cdc2-438e-abbd-162a2fbf4390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:53.630461 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b601dd2-cdc2-438e-abbd-162a2fbf4390 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:53.630754 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b601dd2-cdc2-438e-abbd-162a2fbf4390 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:53.631077 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b601dd2-cdc2-438e-abbd-162a2fbf4390 None admin] PATH_INFO: `/v3/users/1ccfba61dc7d4ad98da496e5800a8589` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:53.632336 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b601dd2-cdc2-438e-abbd-162a2fbf4390 None admin] RBAC: Authorizing `identity:get_user(user_id=1ccfba61dc7d4ad98da496e5800a8589)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:53.634028 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b601dd2-cdc2-438e-abbd-162a2fbf4390 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:53.636828 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 44/87] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Sep 9 11:58:53 2022] GET /identity/v3/users/1ccfba61dc7d4ad98da496e5800a8589 => generated 275 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:58:53.637690 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b601dd2-cdc2-438e-abbd-162a2fbf4390 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:53.648832 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:53.750380 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:53.759401 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b601dd2-cdc2-438e-abbd-162a2fbf4390 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:53.774863 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2183d8-cebd-4505-a45f-f7ccd94d0a5f None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:53.797770 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563f2b7a-1277-4304-ac5d-707329ef3869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:53.798729 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563f2b7a-1277-4304-ac5d-707329ef3869 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:53.799233 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563f2b7a-1277-4304-ac5d-707329ef3869 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:53.799775 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563f2b7a-1277-4304-ac5d-707329ef3869 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:53.821606 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d13094f-226c-43e5-88e1-f828c12d5311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:53.822296 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d13094f-226c-43e5-88e1-f828c12d5311 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:53.822584 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d13094f-226c-43e5-88e1-f828c12d5311 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:53.822861 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d13094f-226c-43e5-88e1-f828c12d5311 None admin] PATH_INFO: `/v3/projects/f8cf6274da2e456eb20ee0ba0d187a83` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:53.824052 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d13094f-226c-43e5-88e1-f828c12d5311 None admin] RBAC: Authorizing `identity:get_project(project_id=f8cf6274da2e456eb20ee0ba0d187a83)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:53.826097 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d13094f-226c-43e5-88e1-f828c12d5311 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:53.828530 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 44/88] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Sep 9 11:58:53 2022] GET /identity/v3/projects/f8cf6274da2e456eb20ee0ba0d187a83 => generated 298 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:58:53.829091 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d13094f-226c-43e5-88e1-f828c12d5311 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:53.891368 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563f2b7a-1277-4304-ac5d-707329ef3869 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:53.893258 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563f2b7a-1277-4304-ac5d-707329ef3869 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:53.922573 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:53.954735 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d13094f-226c-43e5-88e1-f828c12d5311 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:53.992281 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383da395-2768-4e5e-b160-4dcb23e834b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:53.993315 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383da395-2768-4e5e-b160-4dcb23e834b6 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:53.993788 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383da395-2768-4e5e-b160-4dcb23e834b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:53.994305 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383da395-2768-4e5e-b160-4dcb23e834b6 None admin] PATH_INFO: `/v3/projects/f8cf6274da2e456eb20ee0ba0d187a83` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:53.995657 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383da395-2768-4e5e-b160-4dcb23e834b6 None admin] RBAC: Authorizing `identity:get_project(project_id=f8cf6274da2e456eb20ee0ba0d187a83)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:53.997365 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383da395-2768-4e5e-b160-4dcb23e834b6 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:54.003002 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 45/89] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Sep 9 11:58:53 2022] GET /identity/v3/projects/f8cf6274da2e456eb20ee0ba0d187a83 => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:58:54.004026 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383da395-2768-4e5e-b160-4dcb23e834b6 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:54.008770 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:54.023276 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 45/90] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 9 11:58:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 11:58:54.024535 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563f2b7a-1277-4304-ac5d-707329ef3869 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:54.122087 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383da395-2768-4e5e-b160-4dcb23e834b6 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:54.153932 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563f2b7a-1277-4304-ac5d-707329ef3869 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:54.162326 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa0418f-2993-403b-9bc9-09d9fad0033f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:54.163066 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa0418f-2993-403b-9bc9-09d9fad0033f None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:54.163417 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa0418f-2993-403b-9bc9-09d9fad0033f None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:54.163780 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa0418f-2993-403b-9bc9-09d9fad0033f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:54.181605 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:54.198746 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1002e800-85c2-407f-ab90-e1d7ad2e6e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:54.199502 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1002e800-85c2-407f-ab90-e1d7ad2e6e0a None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:54.199888 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1002e800-85c2-407f-ab90-e1d7ad2e6e0a None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:54.200174 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1002e800-85c2-407f-ab90-e1d7ad2e6e0a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:54.200971 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1002e800-85c2-407f-ab90-e1d7ad2e6e0a None admin] RBAC: Adding query filter params (role.id=eecd2f160dd844f5906f3d345780c501, scope.project.id=f8cf6274da2e456eb20ee0ba0d187a83, user.id=1ccfba61dc7d4ad98da496e5800a8589) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:58:54.201346 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1002e800-85c2-407f-ab90-e1d7ad2e6e0a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:54.203124 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1002e800-85c2-407f-ab90-e1d7ad2e6e0a None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:54.210441 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 46/91] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Sep 9 11:58:54 2022] GET /identity/v3/role_assignments?user.id=1ccfba61dc7d4ad98da496e5800a8589&scope.project.id=f8cf6274da2e456eb20ee0ba0d187a83&role.id=eecd2f160dd844f5906f3d345780c501 => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:58:54.211406 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1002e800-85c2-407f-ab90-e1d7ad2e6e0a None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:54.266325 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa0418f-2993-403b-9bc9-09d9fad0033f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:54.269035 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa0418f-2993-403b-9bc9-09d9fad0033f None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:54.282350 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:54.339129 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1002e800-85c2-407f-ab90-e1d7ad2e6e0a None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:54.411099 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c0fe3c-4a42-45aa-a016-c9aecd33b594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:54.411678 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c0fe3c-4a42-45aa-a016-c9aecd33b594 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:54.411985 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c0fe3c-4a42-45aa-a016-c9aecd33b594 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:54.412449 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c0fe3c-4a42-45aa-a016-c9aecd33b594 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:54.413395 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c0fe3c-4a42-45aa-a016-c9aecd33b594 None admin] RBAC: Adding query filter params (role.id=3f2ca33902a04b6dbc2ef7d06d7289d3, scope.project.id=f8cf6274da2e456eb20ee0ba0d187a83, user.id=136011715ec34418adc0f536f6c7357c) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:58:54.413703 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c0fe3c-4a42-45aa-a016-c9aecd33b594 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:54.416386 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c0fe3c-4a42-45aa-a016-c9aecd33b594 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:54.433107 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 47/92] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Sep 9 11:58:54 2022] GET /identity/v3/role_assignments?user.id=136011715ec34418adc0f536f6c7357c&scope.project.id=f8cf6274da2e456eb20ee0ba0d187a83&role.id=3f2ca33902a04b6dbc2ef7d06d7289d3 => generated 259 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:58:54.434010 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c0fe3c-4a42-45aa-a016-c9aecd33b594 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:54.450884 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 46/93] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 9 11:58:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 11:58:54.457002 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa0418f-2993-403b-9bc9-09d9fad0033f None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:54.552162 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:54.658704 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c0fe3c-4a42-45aa-a016-c9aecd33b594 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:54.681615 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa0418f-2993-403b-9bc9-09d9fad0033f None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:54.701675 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:54.723613 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586130ea-c8f9-4f3e-8f8c-a66ca5ae8756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:54.724326 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586130ea-c8f9-4f3e-8f8c-a66ca5ae8756 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:54.724607 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586130ea-c8f9-4f3e-8f8c-a66ca5ae8756 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:54.724888 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586130ea-c8f9-4f3e-8f8c-a66ca5ae8756 None admin] PATH_INFO: `/v3/projects/f8cf6274da2e456eb20ee0ba0d187a83` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:54.726688 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586130ea-c8f9-4f3e-8f8c-a66ca5ae8756 None admin] RBAC: Authorizing `identity:get_project(project_id=f8cf6274da2e456eb20ee0ba0d187a83)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:54.728178 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586130ea-c8f9-4f3e-8f8c-a66ca5ae8756 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:54.731570 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 48/94] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Sep 9 11:58:54 2022] GET /identity/v3/projects/f8cf6274da2e456eb20ee0ba0d187a83 => generated 298 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:58:54.741160 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586130ea-c8f9-4f3e-8f8c-a66ca5ae8756 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:54.753636 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f8e71f-0009-444c-af4a-fdf94b65b4c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:54.754283 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f8e71f-0009-444c-af4a-fdf94b65b4c7 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:54.754560 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f8e71f-0009-444c-af4a-fdf94b65b4c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:54.754831 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f8e71f-0009-444c-af4a-fdf94b65b4c7 None admin] PATH_INFO: `/v3/projects/2d1d07ce59064a289d3ee8acb5552db9` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:54.764207 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f8e71f-0009-444c-af4a-fdf94b65b4c7 None admin] RBAC: Authorizing `identity:get_project(project_id=2d1d07ce59064a289d3ee8acb5552db9)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:54.765690 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f8e71f-0009-444c-af4a-fdf94b65b4c7 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:54.769752 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 47/95] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Sep 9 11:58:54 2022] GET /identity/v3/projects/2d1d07ce59064a289d3ee8acb5552db9 => generated 312 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:58:54.777509 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f8e71f-0009-444c-af4a-fdf94b65b4c7 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:54.934947 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586130ea-c8f9-4f3e-8f8c-a66ca5ae8756 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:54.955757 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f8e71f-0009-444c-af4a-fdf94b65b4c7 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:54.990709 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:54.999969 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bfc1c5-587a-4c96-9594-675a6508a08d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:55.000758 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bfc1c5-587a-4c96-9594-675a6508a08d None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:55.000994 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bfc1c5-587a-4c96-9594-675a6508a08d None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:55.001269 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bfc1c5-587a-4c96-9594-675a6508a08d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:55.001998 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bfc1c5-587a-4c96-9594-675a6508a08d None admin] RBAC: Adding query filter params (role.id=a7c71d2d9914408bb9953f85329d1e80, scope.project.id=f8cf6274da2e456eb20ee0ba0d187a83, user.id=1ccfba61dc7d4ad98da496e5800a8589) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:58:55.002301 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bfc1c5-587a-4c96-9594-675a6508a08d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:55.003758 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bfc1c5-587a-4c96-9594-675a6508a08d None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:55.014520 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 49/96] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Sep 9 11:58:54 2022] GET /identity/v3/role_assignments?user.id=1ccfba61dc7d4ad98da496e5800a8589&scope.project.id=f8cf6274da2e456eb20ee0ba0d187a83&role.id=a7c71d2d9914408bb9953f85329d1e80 => generated 259 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:58:55.028905 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9c2192-1055-4373-809a-8b1f93210344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:55.029626 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9c2192-1055-4373-809a-8b1f93210344 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:55.029905 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9c2192-1055-4373-809a-8b1f93210344 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:55.030177 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9c2192-1055-4373-809a-8b1f93210344 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:55.030934 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9c2192-1055-4373-809a-8b1f93210344 None admin] RBAC: Adding query filter params (role.id=eecd2f160dd844f5906f3d345780c501, scope.project.id=2d1d07ce59064a289d3ee8acb5552db9, user.id=1ccfba61dc7d4ad98da496e5800a8589) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:58:55.031248 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9c2192-1055-4373-809a-8b1f93210344 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:55.032703 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9c2192-1055-4373-809a-8b1f93210344 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:55.043608 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 48/97] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Sep 9 11:58:54 2022] GET /identity/v3/role_assignments?user.id=1ccfba61dc7d4ad98da496e5800a8589&scope.project.id=2d1d07ce59064a289d3ee8acb5552db9&role.id=eecd2f160dd844f5906f3d345780c501 => generated 259 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:58:55.521708 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d1a01b-76f1-41cb-ba7c-b26f759a7440 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:55.522084 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d1a01b-76f1-41cb-ba7c-b26f759a7440 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:55.522278 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d1a01b-76f1-41cb-ba7c-b26f759a7440 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:55.523827 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 50/98] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 11:58:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:58:55.536089 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88ee311-ede9-4932-ba22-90bd72101b42 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:55.536376 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88ee311-ede9-4932-ba22-90bd72101b42 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:55.537318 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88ee311-ede9-4932-ba22-90bd72101b42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:55.560289 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-f88ee311-ede9-4932-ba22-90bd72101b42 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:58:55.561516 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-f88ee311-ede9-4932-ba22-90bd72101b42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:55.655844 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 49/99] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 11:58:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:58:55.839008 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ae2dfe-f94a-472d-9274-ff66ec869b72 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:55.839490 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ae2dfe-f94a-472d-9274-ff66ec869b72 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:55.839566 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ae2dfe-f94a-472d-9274-ff66ec869b72 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:55.840871 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 51/100] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 11:58:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:58:55.855467 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb748548-a822-4b19-a2fb-5bfbc7a845a4 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:55.855755 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb748548-a822-4b19-a2fb-5bfbc7a845a4 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:55.856037 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb748548-a822-4b19-a2fb-5bfbc7a845a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:55.879072 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-bb748548-a822-4b19-a2fb-5bfbc7a845a4 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:58:55.880931 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-bb748548-a822-4b19-a2fb-5bfbc7a845a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:55.972614 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 50/101] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 11:58:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:58:56.249305 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ae2dfe-f94a-472d-9274-ff66ec869b72 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:56.285690 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb748548-a822-4b19-a2fb-5bfbc7a845a4 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:56.310948 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:56.364968 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:56.388555 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ae2dfe-f94a-472d-9274-ff66ec869b72 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:56.452045 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee1e37c-6d41-46da-9385-b3a3becfc0fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:56.452839 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee1e37c-6d41-46da-9385-b3a3becfc0fb None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:56.453124 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee1e37c-6d41-46da-9385-b3a3becfc0fb None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:56.453398 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee1e37c-6d41-46da-9385-b3a3becfc0fb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:56.475575 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee1e37c-6d41-46da-9385-b3a3becfc0fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:58:56.480988 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-7ee1e37c-6d41-46da-9385-b3a3becfc0fb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 11:58:56.480988 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:58:56.480988 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:58:56.480988 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:58:56.480988 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:58:56.480988 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:58:56.480988 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:58:56.480988 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:58:56.480988 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:58:56.480988 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:58:56.480988 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:58:56.480988 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:58:56.480988 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:58:56.480988 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:58:56.480988 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:58:56.480988 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:58:56.481851 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:58:56.481851 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:58:56.481851 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:58:56.481851 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:58:56.481851 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:58:56.481851 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:58:56.481851 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:58:56.481851 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:58:56.481851 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:58:56.481851 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:58:56.481851 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:58:56.481851 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 11:58:56.481851 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:58:56.481851 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:58:56.481851 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:58:56.481851 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:58:56.482599 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:58:56.482599 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:58:56.482599 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:58:56.482599 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:58:56.482599 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:58:56.482599 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:58:56.482599 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:58:56.482599 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:58:56.482599 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:58:56.482599 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:58:56.482599 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 11:58:56.482599 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 11:58:56.482599 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 52/102] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 9 11:58:56 2022] GET /identity/v3/roles/reader => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:58:56.489270 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee1e37c-6d41-46da-9385-b3a3becfc0fb None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:56.530460 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb748548-a822-4b19-a2fb-5bfbc7a845a4 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:56.633070 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ee818d-7bde-4452-99a7-0bbc7d533960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:56.633850 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ee818d-7bde-4452-99a7-0bbc7d533960 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:56.634035 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ee818d-7bde-4452-99a7-0bbc7d533960 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:56.634335 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ee818d-7bde-4452-99a7-0bbc7d533960 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:56.639865 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ee818d-7bde-4452-99a7-0bbc7d533960 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:56.647087 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ee818d-7bde-4452-99a7-0bbc7d533960 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:56.648828 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 51/103] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 11:58:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 364 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:58:56.653480 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee1e37c-6d41-46da-9385-b3a3becfc0fb None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:56.659978 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ee818d-7bde-4452-99a7-0bbc7d533960 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:56.679320 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:56.712609 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dd52cd-2f0e-4dac-bd93-81ca1e561462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:56.713238 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dd52cd-2f0e-4dac-bd93-81ca1e561462 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:56.713584 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dd52cd-2f0e-4dac-bd93-81ca1e561462 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:56.713863 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dd52cd-2f0e-4dac-bd93-81ca1e561462 None admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:56.714582 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dd52cd-2f0e-4dac-bd93-81ca1e561462 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:58:56.714890 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dd52cd-2f0e-4dac-bd93-81ca1e561462 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:56.716336 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dd52cd-2f0e-4dac-bd93-81ca1e561462 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:56.738010 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 53/104] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Sep 9 11:58:56 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:58:56.744519 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dd52cd-2f0e-4dac-bd93-81ca1e561462 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:56.836182 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ee818d-7bde-4452-99a7-0bbc7d533960 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:56.839222 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:56.907142 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c9cc19-5978-4384-9f27-b80063540859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:56.908166 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c9cc19-5978-4384-9f27-b80063540859 None admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:56.908651 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c9cc19-5978-4384-9f27-b80063540859 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:56.909123 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c9cc19-5978-4384-9f27-b80063540859 None admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:56.909977 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c9cc19-5978-4384-9f27-b80063540859 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:56.911757 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c9cc19-5978-4384-9f27-b80063540859 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:56.923320 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dd52cd-2f0e-4dac-bd93-81ca1e561462 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:56.973949 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 52/105] 10.222.0.49 () {60 vars in 1141 bytes} [Fri Sep 9 11:58:56 2022] POST /identity/v3/projects => generated 302 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:58:56.989203 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e67981-66c1-4e64-ae0f-aea99e9dddc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:56.990409 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e67981-66c1-4e64-ae0f-aea99e9dddc5 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:56.990739 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e67981-66c1-4e64-ae0f-aea99e9dddc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:56.991039 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e67981-66c1-4e64-ae0f-aea99e9dddc5 None admin] PATH_INFO: `/v3/users/4b07995ae7d54b759029ad81aa305946` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:57.018328 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e67981-66c1-4e64-ae0f-aea99e9dddc5 None admin] RBAC: Authorizing `identity:get_user(user_id=4b07995ae7d54b759029ad81aa305946)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:57.019843 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e67981-66c1-4e64-ae0f-aea99e9dddc5 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:57.022008 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 54/106] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Sep 9 11:58:56 2022] GET /identity/v3/users/4b07995ae7d54b759029ad81aa305946 => generated 289 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:58:57.027348 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c9cc19-5978-4384-9f27-b80063540859 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:57.190428 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c9cc19-5978-4384-9f27-b80063540859 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:57.239655 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e687bd-7cee-4dfd-9fd9-760ecf1f428b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:57.241243 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e687bd-7cee-4dfd-9fd9-760ecf1f428b None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:57.241727 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e687bd-7cee-4dfd-9fd9-760ecf1f428b None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:57.242197 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e687bd-7cee-4dfd-9fd9-760ecf1f428b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:57.269416 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:57.321060 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e687bd-7cee-4dfd-9fd9-760ecf1f428b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:57.323049 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e687bd-7cee-4dfd-9fd9-760ecf1f428b None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:57.326291 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de42abe-653b-46ab-beb3-53e5ecefe4dd None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:57.326586 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de42abe-653b-46ab-beb3-53e5ecefe4dd None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:57.326864 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de42abe-653b-46ab-beb3-53e5ecefe4dd None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:57.328308 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 55/107] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 11:58:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:58:57.338632 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818c18b2-f2c6-4dfd-bc63-0e038abc9d50 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:57.338974 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818c18b2-f2c6-4dfd-bc63-0e038abc9d50 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:57.339227 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818c18b2-f2c6-4dfd-bc63-0e038abc9d50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:57.376074 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-818c18b2-f2c6-4dfd-bc63-0e038abc9d50 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:58:57.377332 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-818c18b2-f2c6-4dfd-bc63-0e038abc9d50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:57.411206 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:57.468944 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 53/108] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 9 11:58:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 11:58:57.475320 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e687bd-7cee-4dfd-9fd9-760ecf1f428b None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:57.477024 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 56/109] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 11:58:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:58:57.539631 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:57.589001 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e687bd-7cee-4dfd-9fd9-760ecf1f428b None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:57.634852 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6e67f8-8efd-4f22-a352-5b734afb7e9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:57.636345 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6e67f8-8efd-4f22-a352-5b734afb7e9d None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:57.636832 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6e67f8-8efd-4f22-a352-5b734afb7e9d None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:57.637247 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde24ca9-a3d7-4416-8615-6e0dcdcecee8 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:57.637459 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6e67f8-8efd-4f22-a352-5b734afb7e9d None admin] PATH_INFO: `/v3/projects/f8cf6274da2e456eb20ee0ba0d187a83` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:57.637936 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde24ca9-a3d7-4416-8615-6e0dcdcecee8 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:57.638411 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde24ca9-a3d7-4416-8615-6e0dcdcecee8 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:57.638672 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6e67f8-8efd-4f22-a352-5b734afb7e9d None admin] RBAC: Authorizing `identity:get_project(project_id=f8cf6274da2e456eb20ee0ba0d187a83)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:57.640170 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 57/110] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 11:58:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 11:58:57.640293 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6e67f8-8efd-4f22-a352-5b734afb7e9d None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:57.644798 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 54/111] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Sep 9 11:58:57 2022] GET /identity/v3/projects/f8cf6274da2e456eb20ee0ba0d187a83 => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:58:57.648323 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7e3c37-392a-44fa-899d-35c6f4888063 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:57.648807 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7e3c37-392a-44fa-899d-35c6f4888063 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:57.649276 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7e3c37-392a-44fa-899d-35c6f4888063 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:57.650720 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6e67f8-8efd-4f22-a352-5b734afb7e9d None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:57.687893 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-0c7e3c37-392a-44fa-899d-35c6f4888063 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:58:57.689681 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-0c7e3c37-392a-44fa-899d-35c6f4888063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:57.800549 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6e67f8-8efd-4f22-a352-5b734afb7e9d None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:57.822566 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 58/112] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 11:58:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:58:57.841457 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:57.855791 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16652096-4ccc-4b13-b69c-20fd9645b02d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:57.856413 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16652096-4ccc-4b13-b69c-20fd9645b02d None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:57.856700 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16652096-4ccc-4b13-b69c-20fd9645b02d None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:57.856976 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16652096-4ccc-4b13-b69c-20fd9645b02d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:57.857891 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16652096-4ccc-4b13-b69c-20fd9645b02d None admin] RBAC: Adding query filter params (role.id=b1e7a1e6ddc54da7ba995fb55fd93016, scope.project.id=f8cf6274da2e456eb20ee0ba0d187a83, user.id=4b07995ae7d54b759029ad81aa305946) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:58:57.860753 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16652096-4ccc-4b13-b69c-20fd9645b02d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:57.862253 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16652096-4ccc-4b13-b69c-20fd9645b02d None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:57.875162 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 55/113] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Sep 9 11:58:57 2022] GET /identity/v3/role_assignments?user.id=4b07995ae7d54b759029ad81aa305946&scope.project.id=f8cf6274da2e456eb20ee0ba0d187a83&role.id=b1e7a1e6ddc54da7ba995fb55fd93016 => generated 259 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:58:57.905704 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7e3c37-392a-44fa-899d-35c6f4888063 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:58.023730 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:58.031948 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7e3c37-392a-44fa-899d-35c6f4888063 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:58.077124 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaec257-9693-411f-a48f-d5fd4ff0b52a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:58.077757 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaec257-9693-411f-a48f-d5fd4ff0b52a None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:58.078046 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaec257-9693-411f-a48f-d5fd4ff0b52a None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:58.078322 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaec257-9693-411f-a48f-d5fd4ff0b52a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:58.094728 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaec257-9693-411f-a48f-d5fd4ff0b52a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:58:58.100694 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-6eaec257-9693-411f-a48f-d5fd4ff0b52a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:58:58.100694 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:58:58.100694 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:58:58.100694 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:58:58.100694 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:58:58.100694 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:58:58.100694 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:58:58.100694 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:58:58.100694 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:58:58.100694 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:58:58.100694 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:58:58.100694 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:58:58.100694 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:58:58.100694 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:58:58.100694 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:58:58.100694 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:58:58.101643 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:58:58.101643 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:58:58.101643 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:58:58.101643 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:58:58.101643 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:58:58.101643 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:58:58.101643 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:58:58.101643 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:58:58.101643 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:58:58.101643 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:58:58.101643 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:58:58.101643 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 11:58:58.101643 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:58:58.101643 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:58:58.101643 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:58:58.101643 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:58:58.102402 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:58:58.102402 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:58:58.102402 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:58:58.102402 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:58:58.102402 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:58:58.102402 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:58:58.102402 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:58:58.102402 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:58:58.102402 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:58:58.102402 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:58:58.102402 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:58:58.102402 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 11:58:58.102402 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 59/114] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Sep 9 11:58:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:58:58.107987 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16652096-4ccc-4b13-b69c-20fd9645b02d None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:58.229701 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16652096-4ccc-4b13-b69c-20fd9645b02d None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:58.276401 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06f2e4d-ab65-4805-ae46-06e03b809092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:58.277031 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06f2e4d-ab65-4805-ae46-06e03b809092 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:58.277314 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06f2e4d-ab65-4805-ae46-06e03b809092 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:58.277592 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06f2e4d-ab65-4805-ae46-06e03b809092 None admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:58.278278 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06f2e4d-ab65-4805-ae46-06e03b809092 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:58:58.278584 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06f2e4d-ab65-4805-ae46-06e03b809092 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:58.280026 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06f2e4d-ab65-4805-ae46-06e03b809092 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:58.290444 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 56/115] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 9 11:58:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:58:58.296398 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaec257-9693-411f-a48f-d5fd4ff0b52a None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:58.412363 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaec257-9693-411f-a48f-d5fd4ff0b52a None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:58.441521 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06f2e4d-ab65-4805-ae46-06e03b809092 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:58.488104 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8535553-9b7f-4a2a-b4c1-037b43523085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:58.489102 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8535553-9b7f-4a2a-b4c1-037b43523085 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:58.489393 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8535553-9b7f-4a2a-b4c1-037b43523085 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:58.491371 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8535553-9b7f-4a2a-b4c1-037b43523085 None admin] PATH_INFO: `/v3/users/136011715ec34418adc0f536f6c7357c` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:58.492770 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8535553-9b7f-4a2a-b4c1-037b43523085 None admin] RBAC: Authorizing `identity:get_user(user_id=136011715ec34418adc0f536f6c7357c)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:58.494277 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8535553-9b7f-4a2a-b4c1-037b43523085 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:58.495967 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 60/116] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Sep 9 11:58:58 2022] GET /identity/v3/users/136011715ec34418adc0f536f6c7357c => generated 247 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:58:58.499007 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f321c6d9-718a-4485-8f59-627d96255991 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:58.500344 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f321c6d9-718a-4485-8f59-627d96255991 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:58.500633 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f321c6d9-718a-4485-8f59-627d96255991 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:58.504138 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 61/117] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 11:58:58 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:58:58.504892 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f321c6d9-718a-4485-8f59-627d96255991 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:58.548787 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:58.558049 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:58.635486 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06f2e4d-ab65-4805-ae46-06e03b809092 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:58.679797 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e801007-54a3-44be-9450-1fb9149418f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:58.680282 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f321c6d9-718a-4485-8f59-627d96255991 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:58.680855 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e801007-54a3-44be-9450-1fb9149418f0 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:58.681284 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e801007-54a3-44be-9450-1fb9149418f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:58.681704 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e801007-54a3-44be-9450-1fb9149418f0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:58.693727 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e801007-54a3-44be-9450-1fb9149418f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:58:58.700443 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-2e801007-54a3-44be-9450-1fb9149418f0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 11:58:58.700443 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:58:58.700443 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:58:58.700443 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:58:58.700443 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:58:58.700443 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:58:58.700443 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:58:58.700443 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:58:58.700443 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:58:58.700443 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:58:58.700443 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:58:58.700443 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:58:58.700443 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:58:58.700443 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:58:58.700443 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:58:58.700443 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:58:58.701423 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:58:58.701423 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:58:58.701423 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:58:58.701423 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:58:58.701423 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:58:58.701423 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:58:58.701423 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:58:58.701423 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:58:58.701423 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:58:58.701423 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:58:58.701423 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:58:58.701423 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 11:58:58.701423 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:58:58.701423 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:58:58.701423 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:58:58.701423 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:58:58.702179 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:58:58.702179 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:58:58.702179 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:58:58.702179 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:58:58.702179 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:58:58.702179 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:58:58.702179 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:58:58.702179 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:58:58.702179 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:58:58.702179 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:58:58.702179 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 11:58:58.702179 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 11:58:58.704469 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 57/118] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 9 11:58:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:58:58.705833 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebfba56-8d76-47ee-9bd0-8f0247c6d7b6 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:58.706326 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebfba56-8d76-47ee-9bd0-8f0247c6d7b6 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:58.706811 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebfba56-8d76-47ee-9bd0-8f0247c6d7b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:58.721472 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d8ad9d-9228-4384-a94b-47b990d82cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:58.722154 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d8ad9d-9228-4384-a94b-47b990d82cd0 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:58.722487 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d8ad9d-9228-4384-a94b-47b990d82cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:58.722767 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d8ad9d-9228-4384-a94b-47b990d82cd0 None admin] PATH_INFO: `/v3/projects/f8cf6274da2e456eb20ee0ba0d187a83` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:58.724053 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d8ad9d-9228-4384-a94b-47b990d82cd0 None admin] RBAC: Authorizing `identity:get_project(project_id=f8cf6274da2e456eb20ee0ba0d187a83)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:58.726139 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d8ad9d-9228-4384-a94b-47b990d82cd0 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:58.727601 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-0ebfba56-8d76-47ee-9bd0-8f0247c6d7b6 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:58:58.728321 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 62/119] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Sep 9 11:58:58 2022] GET /identity/v3/projects/f8cf6274da2e456eb20ee0ba0d187a83 => generated 298 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:58:58.729343 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-0ebfba56-8d76-47ee-9bd0-8f0247c6d7b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:58.729613 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648c3c67-8bd3-4187-bb9d-fd84f3d88fab None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:58.729944 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648c3c67-8bd3-4187-bb9d-fd84f3d88fab None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:58.730224 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648c3c67-8bd3-4187-bb9d-fd84f3d88fab None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:58.731678 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 63/120] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 11:58:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:58:58.735405 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648c3c67-8bd3-4187-bb9d-fd84f3d88fab None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:58.809693 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 58/121] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 11:58:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:58:58.810866 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebfba56-8d76-47ee-9bd0-8f0247c6d7b6 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:58.848875 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:58.869478 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:58.875727 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648c3c67-8bd3-4187-bb9d-fd84f3d88fab None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:58.943804 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a336dbcd-dce6-40ee-a502-316617328c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:58.944791 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a336dbcd-dce6-40ee-a502-316617328c42 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:58.945283 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a336dbcd-dce6-40ee-a502-316617328c42 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:58.946853 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a336dbcd-dce6-40ee-a502-316617328c42 None admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:58.947821 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a336dbcd-dce6-40ee-a502-316617328c42 None admin] RBAC: Adding query filter params (name=member) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:58:58.948724 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a336dbcd-dce6-40ee-a502-316617328c42 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:58.950758 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a336dbcd-dce6-40ee-a502-316617328c42 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:58.967711 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 64/122] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Sep 9 11:58:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:58:58.969922 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51621959-fc18-4a7b-8da3-f3b88a15e458 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:58.971401 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51621959-fc18-4a7b-8da3-f3b88a15e458 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:58.972953 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51621959-fc18-4a7b-8da3-f3b88a15e458 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:58.984378 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebfba56-8d76-47ee-9bd0-8f0247c6d7b6 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:59.010054 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-51621959-fc18-4a7b-8da3-f3b88a15e458 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:58:59.011598 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-51621959-fc18-4a7b-8da3-f3b88a15e458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:58:59.048824 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e6213b-d528-4a97-95ac-b366ebaf6b7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:59.049833 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e6213b-d528-4a97-95ac-b366ebaf6b7b None admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:59.050634 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e6213b-d528-4a97-95ac-b366ebaf6b7b None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:59.051717 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e6213b-d528-4a97-95ac-b366ebaf6b7b None admin] PATH_INFO: `/v3/projects/f8cf6274da2e456eb20ee0ba0d187a83/users/136011715ec34418adc0f536f6c7357c/roles/3f2ca33902a04b6dbc2ef7d06d7289d3` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:59.056358 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e6213b-d528-4a97-95ac-b366ebaf6b7b None admin] RBAC: Authorizing `identity:create_grant(project_id=f8cf6274da2e456eb20ee0ba0d187a83, user_id=136011715ec34418adc0f536f6c7357c, role_id=3f2ca33902a04b6dbc2ef7d06d7289d3)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:59.058778 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e6213b-d528-4a97-95ac-b366ebaf6b7b None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:59.086652 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 59/123] 10.222.0.49 () {58 vars in 1439 bytes} [Fri Sep 9 11:58:58 2022] PUT /identity/v3/projects/f8cf6274da2e456eb20ee0ba0d187a83/users/136011715ec34418adc0f536f6c7357c/roles/3f2ca33902a04b6dbc2ef7d06d7289d3 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:58:59.087962 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e6213b-d528-4a97-95ac-b366ebaf6b7b None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:59.153649 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 65/124] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 11:58:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:58:59.155948 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51621959-fc18-4a7b-8da3-f3b88a15e458 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:59.248703 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e6213b-d528-4a97-95ac-b366ebaf6b7b None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:59.264603 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:59.287732 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11e98b5-ae63-46dc-b8f3-bccc54074687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:59.288621 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11e98b5-ae63-46dc-b8f3-bccc54074687 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:59.289047 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11e98b5-ae63-46dc-b8f3-bccc54074687 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:59.289430 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11e98b5-ae63-46dc-b8f3-bccc54074687 None admin] PATH_INFO: `/v3/users/1ccfba61dc7d4ad98da496e5800a8589` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:59.290767 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11e98b5-ae63-46dc-b8f3-bccc54074687 None admin] RBAC: Authorizing `identity:get_user(user_id=1ccfba61dc7d4ad98da496e5800a8589)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:59.292416 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11e98b5-ae63-46dc-b8f3-bccc54074687 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:59.294185 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 60/125] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Sep 9 11:58:59 2022] GET /identity/v3/users/1ccfba61dc7d4ad98da496e5800a8589 => generated 275 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:58:59.299863 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11e98b5-ae63-46dc-b8f3-bccc54074687 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:59.372097 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:59.410435 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51621959-fc18-4a7b-8da3-f3b88a15e458 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:59.428808 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11e98b5-ae63-46dc-b8f3-bccc54074687 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:59.467946 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c7ab47-8989-4018-981b-e1f48aee122d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:59.468287 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2340452-8712-4252-a497-18c114a93acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:59.468598 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c7ab47-8989-4018-981b-e1f48aee122d None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:59.468883 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c7ab47-8989-4018-981b-e1f48aee122d None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:59.469163 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c7ab47-8989-4018-981b-e1f48aee122d None admin] PATH_INFO: `/v3/projects/f8cf6274da2e456eb20ee0ba0d187a83` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:59.469761 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2340452-8712-4252-a497-18c114a93acc None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:59.470266 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2340452-8712-4252-a497-18c114a93acc None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:59.470364 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c7ab47-8989-4018-981b-e1f48aee122d None admin] RBAC: Authorizing `identity:get_project(project_id=f8cf6274da2e456eb20ee0ba0d187a83)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:59.470812 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2340452-8712-4252-a497-18c114a93acc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:59.471950 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c7ab47-8989-4018-981b-e1f48aee122d None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:59.473641 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 61/126] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Sep 9 11:58:59 2022] GET /identity/v3/projects/f8cf6274da2e456eb20ee0ba0d187a83 => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:58:59.480676 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c7ab47-8989-4018-981b-e1f48aee122d None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:59.488544 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2340452-8712-4252-a497-18c114a93acc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:58:59.494871 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-c2340452-8712-4252-a497-18c114a93acc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 11:58:59.494871 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:58:59.494871 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:58:59.494871 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:58:59.494871 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:58:59.494871 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:58:59.494871 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:58:59.494871 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:58:59.494871 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:58:59.494871 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:58:59.494871 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:58:59.494871 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:58:59.494871 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:58:59.494871 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:58:59.494871 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:58:59.494871 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:58:59.495755 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:58:59.495755 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:58:59.495755 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:58:59.495755 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:58:59.495755 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:58:59.495755 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:58:59.495755 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:58:59.495755 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:58:59.495755 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:58:59.495755 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:58:59.495755 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:58:59.495755 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 11:58:59.495755 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:58:59.495755 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:58:59.495755 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:58:59.495755 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:58:59.496496 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:58:59.496496 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:58:59.496496 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:58:59.496496 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:58:59.496496 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:58:59.496496 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:58:59.496496 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:58:59.496496 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:58:59.496496 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:58:59.496496 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:58:59.496496 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 11:58:59.496496 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 11:58:59.499384 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 66/127] 10.222.0.49 () {56 vars in 1114 bytes} [Fri Sep 9 11:58:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:58:59.505998 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2340452-8712-4252-a497-18c114a93acc None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:59.519332 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:59.592858 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c7ab47-8989-4018-981b-e1f48aee122d None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:59.633105 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cb437c-f754-4dc5-bb88-88ce27ffb47e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:59.634069 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cb437c-f754-4dc5-bb88-88ce27ffb47e None admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:59.634358 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cb437c-f754-4dc5-bb88-88ce27ffb47e None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:59.634635 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cb437c-f754-4dc5-bb88-88ce27ffb47e None admin] PATH_INFO: `/v3/projects/f8cf6274da2e456eb20ee0ba0d187a83/users/1ccfba61dc7d4ad98da496e5800a8589/roles/eecd2f160dd844f5906f3d345780c501` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:59.636638 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cb437c-f754-4dc5-bb88-88ce27ffb47e None admin] RBAC: Authorizing `identity:create_grant(project_id=f8cf6274da2e456eb20ee0ba0d187a83, user_id=1ccfba61dc7d4ad98da496e5800a8589, role_id=eecd2f160dd844f5906f3d345780c501)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:59.638181 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cb437c-f754-4dc5-bb88-88ce27ffb47e None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:59.659957 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2340452-8712-4252-a497-18c114a93acc None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:59.673703 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 62/128] 10.222.0.49 () {58 vars in 1439 bytes} [Fri Sep 9 11:58:59 2022] PUT /identity/v3/projects/f8cf6274da2e456eb20ee0ba0d187a83/users/1ccfba61dc7d4ad98da496e5800a8589/roles/eecd2f160dd844f5906f3d345780c501 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 11:58:59.674572 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cb437c-f754-4dc5-bb88-88ce27ffb47e None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:59.706824 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04907445-d81a-422d-aa8f-7b8e927471c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:59.712147 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04907445-d81a-422d-aa8f-7b8e927471c6 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:59.712962 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04907445-d81a-422d-aa8f-7b8e927471c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:59.714336 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04907445-d81a-422d-aa8f-7b8e927471c6 None admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:59.715481 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04907445-d81a-422d-aa8f-7b8e927471c6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:58:59.716249 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04907445-d81a-422d-aa8f-7b8e927471c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:58:59.718829 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04907445-d81a-422d-aa8f-7b8e927471c6 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:58:59.744134 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 67/129] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 9 11:58:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:58:59.751142 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04907445-d81a-422d-aa8f-7b8e927471c6 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:58:59.761660 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:59.895095 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:58:59.901499 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cb437c-f754-4dc5-bb88-88ce27ffb47e None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:59.967901 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53063d03-1d49-4788-b0ca-01bc56a8beb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:58:59.968910 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53063d03-1d49-4788-b0ca-01bc56a8beb8 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:58:59.969388 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53063d03-1d49-4788-b0ca-01bc56a8beb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:58:59.969878 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53063d03-1d49-4788-b0ca-01bc56a8beb8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:58:59.983047 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04907445-d81a-422d-aa8f-7b8e927471c6 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:58:59.996503 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53063d03-1d49-4788-b0ca-01bc56a8beb8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:59:00.002243 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-53063d03-1d49-4788-b0ca-01bc56a8beb8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 11:59:00.002243 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:59:00.002243 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:59:00.002243 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:59:00.002243 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:59:00.002243 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:59:00.002243 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:59:00.002243 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:59:00.002243 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:59:00.002243 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:59:00.002243 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:59:00.002243 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:59:00.002243 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:59:00.002243 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:59:00.002243 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:59:00.002243 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:59:00.003110 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:59:00.003110 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:59:00.003110 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:59:00.003110 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:59:00.003110 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:59:00.003110 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:59:00.003110 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:59:00.003110 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:59:00.003110 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:59:00.003110 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:59:00.003110 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:59:00.003110 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 11:59:00.003110 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:59:00.003110 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:59:00.003110 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:59:00.003110 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:59:00.003902 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:59:00.003902 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:59:00.003902 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:59:00.003902 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:59:00.003902 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:59:00.003902 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:59:00.003902 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:59:00.003902 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:59:00.003902 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:59:00.003902 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:59:00.003902 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 11:59:00.003902 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 11:59:00.007570 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 63/130] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 9 11:58:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:59:00.013734 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53063d03-1d49-4788-b0ca-01bc56a8beb8 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:00.052444 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42753159-7304-4070-835c-370d3912ae1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:00.054342 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42753159-7304-4070-835c-370d3912ae1f None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:00.055070 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42753159-7304-4070-835c-370d3912ae1f None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:00.058159 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42753159-7304-4070-835c-370d3912ae1f None admin] PATH_INFO: `/v3/users/1ccfba61dc7d4ad98da496e5800a8589` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:00.059633 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42753159-7304-4070-835c-370d3912ae1f None admin] RBAC: Authorizing `identity:get_user(user_id=1ccfba61dc7d4ad98da496e5800a8589)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:00.061355 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42753159-7304-4070-835c-370d3912ae1f None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:00.063466 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 68/131] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Sep 9 11:58:59 2022] GET /identity/v3/users/1ccfba61dc7d4ad98da496e5800a8589 => generated 275 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:00.071553 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42753159-7304-4070-835c-370d3912ae1f None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:00.250295 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:00.268976 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53063d03-1d49-4788-b0ca-01bc56a8beb8 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:00.321788 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42753159-7304-4070-835c-370d3912ae1f None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:00.324936 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2b7ee2-b27a-4a7c-a944-6f1585bdee37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:00.325550 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2b7ee2-b27a-4a7c-a944-6f1585bdee37 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:00.325836 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2b7ee2-b27a-4a7c-a944-6f1585bdee37 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:00.326113 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2b7ee2-b27a-4a7c-a944-6f1585bdee37 None admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:00.326947 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2b7ee2-b27a-4a7c-a944-6f1585bdee37 None admin] RBAC: Adding query filter params (name=member) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:59:00.327115 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2b7ee2-b27a-4a7c-a944-6f1585bdee37 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:00.328521 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2b7ee2-b27a-4a7c-a944-6f1585bdee37 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:00.346427 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 64/132] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Sep 9 11:59:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:00.353466 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2b7ee2-b27a-4a7c-a944-6f1585bdee37 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:00.393687 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:00.405266 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6cdada-71c5-45cf-972b-adae11464ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:00.405920 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6cdada-71c5-45cf-972b-adae11464ca3 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:00.406256 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6cdada-71c5-45cf-972b-adae11464ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:00.406539 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6cdada-71c5-45cf-972b-adae11464ca3 None admin] PATH_INFO: `/v3/projects/f8cf6274da2e456eb20ee0ba0d187a83` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:00.407732 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6cdada-71c5-45cf-972b-adae11464ca3 None admin] RBAC: Authorizing `identity:get_project(project_id=f8cf6274da2e456eb20ee0ba0d187a83)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:00.409183 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6cdada-71c5-45cf-972b-adae11464ca3 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:00.410834 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 69/133] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Sep 9 11:59:00 2022] GET /identity/v3/projects/f8cf6274da2e456eb20ee0ba0d187a83 => generated 298 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:00.417703 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6cdada-71c5-45cf-972b-adae11464ca3 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:00.637044 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2b7ee2-b27a-4a7c-a944-6f1585bdee37 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:00.653025 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6cdada-71c5-45cf-972b-adae11464ca3 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:00.690695 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bfad0e-4bce-4239-87da-edb20bf04077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:00.691401 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bfad0e-4bce-4239-87da-edb20bf04077 None admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:00.691676 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bfad0e-4bce-4239-87da-edb20bf04077 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:00.692043 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bfad0e-4bce-4239-87da-edb20bf04077 None admin] PATH_INFO: `/v3/projects/f8cf6274da2e456eb20ee0ba0d187a83/users/1ccfba61dc7d4ad98da496e5800a8589/roles/a7c71d2d9914408bb9953f85329d1e80` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:00.694729 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bfad0e-4bce-4239-87da-edb20bf04077 None admin] RBAC: Authorizing `identity:create_grant(project_id=f8cf6274da2e456eb20ee0ba0d187a83, user_id=1ccfba61dc7d4ad98da496e5800a8589, role_id=a7c71d2d9914408bb9953f85329d1e80)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:00.696349 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bfad0e-4bce-4239-87da-edb20bf04077 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:00.706717 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:00.715422 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 70/134] 10.222.0.49 () {58 vars in 1439 bytes} [Fri Sep 9 11:59:00 2022] PUT /identity/v3/projects/f8cf6274da2e456eb20ee0ba0d187a83/users/1ccfba61dc7d4ad98da496e5800a8589/roles/a7c71d2d9914408bb9953f85329d1e80 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:59:00.718939 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7f66d7-358c-4ecb-a04a-c4706479c1c9 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:00.718939 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7f66d7-358c-4ecb-a04a-c4706479c1c9 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:00.718939 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7f66d7-358c-4ecb-a04a-c4706479c1c9 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:00.719900 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 71/135] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 11:59:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:59:00.723315 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8263a82b-1ef4-40ae-93d9-224864fc4267 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:00.723315 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8263a82b-1ef4-40ae-93d9-224864fc4267 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:00.723315 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8263a82b-1ef4-40ae-93d9-224864fc4267 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:00.723315 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 72/136] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 11:59:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:59:00.731886 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ed8002-85ff-4a67-ab8f-e36511a5d872 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:00.732372 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ed8002-85ff-4a67-ab8f-e36511a5d872 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:00.732816 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ed8002-85ff-4a67-ab8f-e36511a5d872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:00.735653 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f722ddf7-3999-4e23-b6ca-f488a0aae046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:00.736317 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f722ddf7-3999-4e23-b6ca-f488a0aae046 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:00.736602 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f722ddf7-3999-4e23-b6ca-f488a0aae046 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:00.736878 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f722ddf7-3999-4e23-b6ca-f488a0aae046 None admin] PATH_INFO: `/v3/users/1ccfba61dc7d4ad98da496e5800a8589` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:00.743803 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f722ddf7-3999-4e23-b6ca-f488a0aae046 None admin] RBAC: Authorizing `identity:get_user(user_id=1ccfba61dc7d4ad98da496e5800a8589)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:00.745416 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f722ddf7-3999-4e23-b6ca-f488a0aae046 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:00.747509 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 65/137] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Sep 9 11:59:00 2022] GET /identity/v3/users/1ccfba61dc7d4ad98da496e5800a8589 => generated 275 bytes in 394 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:00.749372 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e153a2-21d2-4d39-9612-013555e59cde None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:00.749372 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e153a2-21d2-4d39-9612-013555e59cde None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:00.749372 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e153a2-21d2-4d39-9612-013555e59cde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:00.762227 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-b2ed8002-85ff-4a67-ab8f-e36511a5d872 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:59:00.763969 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-b2ed8002-85ff-4a67-ab8f-e36511a5d872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:00.780640 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-a2e153a2-21d2-4d39-9612-013555e59cde None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:59:00.781838 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-a2e153a2-21d2-4d39-9612-013555e59cde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:00.857864 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 73/138] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 11:59:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 11:59:00.858827 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ed8002-85ff-4a67-ab8f-e36511a5d872 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:00.877455 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 66/139] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 11:59:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:59:00.975013 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ed8002-85ff-4a67-ab8f-e36511a5d872 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:01.004720 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:01.011574 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b184b4d-4759-45a0-987f-23a7b68d8f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:01.012222 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b184b4d-4759-45a0-987f-23a7b68d8f8f None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:01.012505 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b184b4d-4759-45a0-987f-23a7b68d8f8f None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:01.012782 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b184b4d-4759-45a0-987f-23a7b68d8f8f None admin] PATH_INFO: `/v3/projects/2d1d07ce59064a289d3ee8acb5552db9` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:01.013939 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b184b4d-4759-45a0-987f-23a7b68d8f8f None admin] RBAC: Authorizing `identity:get_project(project_id=2d1d07ce59064a289d3ee8acb5552db9)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:01.015617 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b184b4d-4759-45a0-987f-23a7b68d8f8f None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:01.017264 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 74/140] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Sep 9 11:59:00 2022] GET /identity/v3/projects/2d1d07ce59064a289d3ee8acb5552db9 => generated 312 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:01.023979 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e153a2-21d2-4d39-9612-013555e59cde None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:01.044826 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:01.152635 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e153a2-21d2-4d39-9612-013555e59cde None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:01.204067 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b184b4d-4759-45a0-987f-23a7b68d8f8f None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:01.204688 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706aebbe-966e-49e8-8a05-2e2b2796549e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:01.205375 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706aebbe-966e-49e8-8a05-2e2b2796549e None admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:01.205715 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706aebbe-966e-49e8-8a05-2e2b2796549e None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:01.205991 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706aebbe-966e-49e8-8a05-2e2b2796549e None admin] PATH_INFO: `/v3/projects/2d1d07ce59064a289d3ee8acb5552db9/users/1ccfba61dc7d4ad98da496e5800a8589/roles/eecd2f160dd844f5906f3d345780c501` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:01.209606 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706aebbe-966e-49e8-8a05-2e2b2796549e None admin] RBAC: Authorizing `identity:create_grant(project_id=2d1d07ce59064a289d3ee8acb5552db9, user_id=1ccfba61dc7d4ad98da496e5800a8589, role_id=eecd2f160dd844f5906f3d345780c501)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:01.211165 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706aebbe-966e-49e8-8a05-2e2b2796549e None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:01.233150 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 67/141] 10.222.0.49 () {58 vars in 1439 bytes} [Fri Sep 9 11:59:01 2022] PUT /identity/v3/projects/2d1d07ce59064a289d3ee8acb5552db9/users/1ccfba61dc7d4ad98da496e5800a8589/roles/eecd2f160dd844f5906f3d345780c501 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:59:01.319676 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b184b4d-4759-45a0-987f-23a7b68d8f8f None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:01.357431 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d822a980-0d4b-4e9c-bcfb-52864b7a0158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:01.358113 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d822a980-0d4b-4e9c-bcfb-52864b7a0158 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:01.358389 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d822a980-0d4b-4e9c-bcfb-52864b7a0158 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:01.358657 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d822a980-0d4b-4e9c-bcfb-52864b7a0158 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:01.359977 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d822a980-0d4b-4e9c-bcfb-52864b7a0158 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:01.361425 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d822a980-0d4b-4e9c-bcfb-52864b7a0158 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:01.363296 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 75/142] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 11:59:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:59:01.370735 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706aebbe-966e-49e8-8a05-2e2b2796549e None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:01.404265 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d822a980-0d4b-4e9c-bcfb-52864b7a0158 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:01.482951 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:01.525002 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d822a980-0d4b-4e9c-bcfb-52864b7a0158 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:01.563329 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:01.572783 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2491fe-dd55-4322-a8e5-1feba55042d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:01.573828 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2491fe-dd55-4322-a8e5-1feba55042d9 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:01.574112 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2491fe-dd55-4322-a8e5-1feba55042d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:01.574641 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2491fe-dd55-4322-a8e5-1feba55042d9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:01.598111 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2491fe-dd55-4322-a8e5-1feba55042d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:59:01.598497 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706aebbe-966e-49e8-8a05-2e2b2796549e None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:01.603223 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-da2491fe-dd55-4322-a8e5-1feba55042d9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 11:59:01.603223 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:59:01.603223 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:59:01.603223 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:59:01.603223 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:59:01.603223 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:59:01.603223 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:59:01.603223 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:59:01.603223 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:59:01.603223 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:59:01.603223 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:59:01.603223 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:59:01.603223 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:59:01.603223 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:59:01.603223 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:59:01.603223 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:59:01.604060 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:59:01.604060 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:59:01.604060 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:59:01.604060 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:59:01.604060 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:59:01.604060 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:59:01.604060 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:59:01.604060 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:59:01.604060 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:59:01.604060 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:59:01.604060 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:59:01.604060 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 11:59:01.604060 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:59:01.604060 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:59:01.604060 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:59:01.604060 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:59:01.604809 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:59:01.604809 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:59:01.604809 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:59:01.604809 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:59:01.604809 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:59:01.604809 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:59:01.604809 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:59:01.604809 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:59:01.604809 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:59:01.604809 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:59:01.604809 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 11:59:01.604809 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 11:59:01.604809 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 76/143] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 9 11:59:01 2022] GET /identity/v3/roles/reader => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:59:01.610881 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2491fe-dd55-4322-a8e5-1feba55042d9 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:01.660099 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19081525-6b19-4a73-8a5e-e22b5cd70cfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:01.660714 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19081525-6b19-4a73-8a5e-e22b5cd70cfd None admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:01.660998 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19081525-6b19-4a73-8a5e-e22b5cd70cfd None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:01.661269 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19081525-6b19-4a73-8a5e-e22b5cd70cfd None admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:01.662233 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19081525-6b19-4a73-8a5e-e22b5cd70cfd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:01.664007 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19081525-6b19-4a73-8a5e-e22b5cd70cfd None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:01.711710 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 68/144] 10.222.0.49 () {60 vars in 1132 bytes} [Fri Sep 9 11:59:01 2022] POST /identity/v3/users => generated 283 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:59:01.777133 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2491fe-dd55-4322-a8e5-1feba55042d9 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:01.816786 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:01.827969 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701921e8-d9f5-4185-ba2c-4fbc1cb6ad58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:01.828595 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701921e8-d9f5-4185-ba2c-4fbc1cb6ad58 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:01.828871 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701921e8-d9f5-4185-ba2c-4fbc1cb6ad58 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:01.829138 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701921e8-d9f5-4185-ba2c-4fbc1cb6ad58 None admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:01.829813 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701921e8-d9f5-4185-ba2c-4fbc1cb6ad58 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:59:01.830112 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701921e8-d9f5-4185-ba2c-4fbc1cb6ad58 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:01.831537 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701921e8-d9f5-4185-ba2c-4fbc1cb6ad58 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:01.841206 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 77/145] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Sep 9 11:59:01 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:01.852100 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19081525-6b19-4a73-8a5e-e22b5cd70cfd None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:01.988013 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19081525-6b19-4a73-8a5e-e22b5cd70cfd None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:02.027052 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:02.035876 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f396c9-b51c-4fbc-9032-fc5d3b257264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:02.036448 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f396c9-b51c-4fbc-9032-fc5d3b257264 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:02.036993 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f396c9-b51c-4fbc-9032-fc5d3b257264 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:02.037321 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f396c9-b51c-4fbc-9032-fc5d3b257264 None admin] PATH_INFO: `/v3/users/4b07995ae7d54b759029ad81aa305946` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:02.039050 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f396c9-b51c-4fbc-9032-fc5d3b257264 None admin] RBAC: Authorizing `identity:get_user(user_id=4b07995ae7d54b759029ad81aa305946)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:02.044520 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f396c9-b51c-4fbc-9032-fc5d3b257264 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:02.046432 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 69/146] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Sep 9 11:59:01 2022] GET /identity/v3/users/4b07995ae7d54b759029ad81aa305946 => generated 289 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:02.051692 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701921e8-d9f5-4185-ba2c-4fbc1cb6ad58 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:02.189849 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701921e8-d9f5-4185-ba2c-4fbc1cb6ad58 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:02.230406 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9df67bf-1119-45fe-bd48-a375f81ded53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:02.231078 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9df67bf-1119-45fe-bd48-a375f81ded53 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:02.231361 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9df67bf-1119-45fe-bd48-a375f81ded53 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:02.231636 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9df67bf-1119-45fe-bd48-a375f81ded53 None admin] PATH_INFO: `/v3/projects/f8cf6274da2e456eb20ee0ba0d187a83` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:02.232770 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9df67bf-1119-45fe-bd48-a375f81ded53 None admin] RBAC: Authorizing `identity:get_project(project_id=f8cf6274da2e456eb20ee0ba0d187a83)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:02.234252 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9df67bf-1119-45fe-bd48-a375f81ded53 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:02.238444 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 78/147] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Sep 9 11:59:02 2022] GET /identity/v3/projects/f8cf6274da2e456eb20ee0ba0d187a83 => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:02.242323 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f396c9-b51c-4fbc-9032-fc5d3b257264 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:02.399129 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f396c9-b51c-4fbc-9032-fc5d3b257264 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:02.440030 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e524fec-fe4c-4e70-ac86-806b0ea763fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:02.441061 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e524fec-fe4c-4e70-ac86-806b0ea763fb None admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:02.441577 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e524fec-fe4c-4e70-ac86-806b0ea763fb None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:02.442067 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e524fec-fe4c-4e70-ac86-806b0ea763fb None admin] PATH_INFO: `/v3/projects/f8cf6274da2e456eb20ee0ba0d187a83/users/4b07995ae7d54b759029ad81aa305946/roles/b1e7a1e6ddc54da7ba995fb55fd93016` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:02.445066 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e524fec-fe4c-4e70-ac86-806b0ea763fb None admin] RBAC: Authorizing `identity:create_grant(project_id=f8cf6274da2e456eb20ee0ba0d187a83, user_id=4b07995ae7d54b759029ad81aa305946, role_id=b1e7a1e6ddc54da7ba995fb55fd93016)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:02.446749 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e524fec-fe4c-4e70-ac86-806b0ea763fb None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:02.464999 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 70/148] 10.222.0.49 () {58 vars in 1439 bytes} [Fri Sep 9 11:59:02 2022] PUT /identity/v3/projects/f8cf6274da2e456eb20ee0ba0d187a83/users/4b07995ae7d54b759029ad81aa305946/roles/b1e7a1e6ddc54da7ba995fb55fd93016 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:59:02.559775 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba256bc-bab8-4155-8383-743d3af948f3 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:02.560273 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba256bc-bab8-4155-8383-743d3af948f3 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:02.560344 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba256bc-bab8-4155-8383-743d3af948f3 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:02.561846 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 79/149] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 11:59:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:59:02.568699 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d08a8da-8574-4c1d-a502-5f2eaef746ab None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:02.569756 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d08a8da-8574-4c1d-a502-5f2eaef746ab None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:02.570495 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d08a8da-8574-4c1d-a502-5f2eaef746ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:02.595166 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-9d08a8da-8574-4c1d-a502-5f2eaef746ab None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:59:02.596750 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-9d08a8da-8574-4c1d-a502-5f2eaef746ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:02.691799 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:02.708726 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 71/150] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 11:59:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:59:03.189284 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fc302e-f654-4e01-aa82-0f64174f7c92 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:03.189652 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fc302e-f654-4e01-aa82-0f64174f7c92 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:03.189848 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fc302e-f654-4e01-aa82-0f64174f7c92 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:03.191205 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 80/151] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 11:59:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:59:03.209351 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4fdf39-f109-4251-9200-6a3d61791d8b None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:03.209658 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4fdf39-f109-4251-9200-6a3d61791d8b None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:03.209898 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4fdf39-f109-4251-9200-6a3d61791d8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:03.227529 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fc302e-f654-4e01-aa82-0f64174f7c92 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:03.235162 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-bd4fdf39-f109-4251-9200-6a3d61791d8b None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:59:03.236365 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-bd4fdf39-f109-4251-9200-6a3d61791d8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:03.315896 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:03.367786 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 72/152] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 11:59:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:59:03.380972 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fc302e-f654-4e01-aa82-0f64174f7c92 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:03.420180 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a95f981-7ded-447f-be5b-f2aa04ac6e5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:03.420867 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a95f981-7ded-447f-be5b-f2aa04ac6e5b None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:03.421144 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a95f981-7ded-447f-be5b-f2aa04ac6e5b None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:03.421412 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a95f981-7ded-447f-be5b-f2aa04ac6e5b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:03.432934 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a95f981-7ded-447f-be5b-f2aa04ac6e5b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:59:03.438437 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-8a95f981-7ded-447f-be5b-f2aa04ac6e5b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:59:03.438437 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:59:03.438437 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:59:03.438437 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:59:03.438437 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:59:03.438437 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:59:03.438437 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:59:03.438437 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:59:03.438437 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:59:03.438437 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:59:03.438437 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:59:03.438437 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:59:03.438437 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:59:03.438437 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:59:03.438437 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:59:03.438437 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:59:03.439766 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:59:03.439766 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:59:03.439766 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:59:03.439766 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:59:03.439766 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:59:03.439766 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:59:03.439766 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:59:03.439766 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:59:03.439766 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:59:03.439766 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:59:03.439766 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:59:03.439766 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 11:59:03.439766 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:59:03.439766 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:59:03.439766 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:59:03.439766 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:59:03.440529 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:59:03.440529 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:59:03.440529 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:59:03.440529 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:59:03.440529 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:59:03.440529 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:59:03.440529 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:59:03.440529 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:59:03.440529 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:59:03.440529 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:59:03.440529 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:59:03.440529 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 11:59:03.440529 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 81/153] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Sep 9 11:59:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:59:03.448327 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4fdf39-f109-4251-9200-6a3d61791d8b None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:03.561679 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4fdf39-f109-4251-9200-6a3d61791d8b None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:03.583921 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:03.603847 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942a4112-c327-4010-8400-a1592d361176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:03.604465 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942a4112-c327-4010-8400-a1592d361176 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:03.604742 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942a4112-c327-4010-8400-a1592d361176 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:03.605014 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942a4112-c327-4010-8400-a1592d361176 None admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:03.605685 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942a4112-c327-4010-8400-a1592d361176 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:59:03.605989 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942a4112-c327-4010-8400-a1592d361176 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:03.607409 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942a4112-c327-4010-8400-a1592d361176 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:03.619147 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 73/154] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 9 11:59:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:03.625043 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a95f981-7ded-447f-be5b-f2aa04ac6e5b None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:03.731739 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a95f981-7ded-447f-be5b-f2aa04ac6e5b None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:03.749155 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:03.768026 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf95ac5-7401-473a-907c-c0ddc64c9751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:03.768666 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf95ac5-7401-473a-907c-c0ddc64c9751 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:03.768943 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf95ac5-7401-473a-907c-c0ddc64c9751 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:03.769262 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf95ac5-7401-473a-907c-c0ddc64c9751 None admin] PATH_INFO: `/v3/users/136011715ec34418adc0f536f6c7357c` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:03.770346 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf95ac5-7401-473a-907c-c0ddc64c9751 None admin] RBAC: Authorizing `identity:get_user(user_id=136011715ec34418adc0f536f6c7357c)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:03.772459 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf95ac5-7401-473a-907c-c0ddc64c9751 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:03.774182 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 82/155] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Sep 9 11:59:03 2022] GET /identity/v3/users/136011715ec34418adc0f536f6c7357c => generated 247 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:03.779958 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942a4112-c327-4010-8400-a1592d361176 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:03.896269 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942a4112-c327-4010-8400-a1592d361176 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:03.937114 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba0ffff-287f-4c83-b93a-cc1239408e5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:03.937731 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba0ffff-287f-4c83-b93a-cc1239408e5f None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:03.938009 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba0ffff-287f-4c83-b93a-cc1239408e5f None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:03.938281 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba0ffff-287f-4c83-b93a-cc1239408e5f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:03.993618 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:04.016258 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba0ffff-287f-4c83-b93a-cc1239408e5f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:04.017736 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba0ffff-287f-4c83-b93a-cc1239408e5f None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:04.028955 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf95ac5-7401-473a-907c-c0ddc64c9751 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:04.186747 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf95ac5-7401-473a-907c-c0ddc64c9751 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:04.196574 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 74/156] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 9 11:59:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 11:59:04.202205 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba0ffff-287f-4c83-b93a-cc1239408e5f None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:04.233895 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa8d844-2c1e-47ae-a3ca-aca37ead3b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:04.234556 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa8d844-2c1e-47ae-a3ca-aca37ead3b99 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:04.234836 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa8d844-2c1e-47ae-a3ca-aca37ead3b99 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:04.235140 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa8d844-2c1e-47ae-a3ca-aca37ead3b99 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:04.266210 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa8d844-2c1e-47ae-a3ca-aca37ead3b99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:59:04.277235 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-2aa8d844-2c1e-47ae-a3ca-aca37ead3b99 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 11:59:04.277235 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:59:04.277235 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:59:04.277235 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:59:04.277235 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:59:04.277235 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:59:04.277235 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:59:04.277235 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:59:04.277235 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:59:04.277235 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:59:04.277235 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:59:04.277235 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:59:04.277235 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:59:04.277235 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:59:04.277235 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:59:04.277235 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:59:04.279524 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:59:04.279524 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:59:04.279524 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:59:04.279524 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:59:04.279524 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:59:04.279524 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:59:04.279524 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:59:04.279524 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:59:04.279524 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:59:04.279524 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:59:04.279524 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:59:04.279524 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 11:59:04.279524 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:59:04.279524 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:59:04.279524 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:59:04.279524 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:59:04.281395 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:59:04.281395 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:59:04.281395 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:59:04.281395 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:59:04.281395 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:59:04.281395 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:59:04.281395 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:59:04.281395 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:59:04.281395 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:59:04.281395 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:59:04.281395 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 11:59:04.281395 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 11:59:04.281395 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 83/157] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 9 11:59:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:59:04.289500 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa8d844-2c1e-47ae-a3ca-aca37ead3b99 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:04.367167 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:04.371622 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:04.407358 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba0ffff-287f-4c83-b93a-cc1239408e5f None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:04.466703 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01334a54-c95a-4246-a67c-3e458b5ce9a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:04.467378 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01334a54-c95a-4246-a67c-3e458b5ce9a9 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:04.467655 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01334a54-c95a-4246-a67c-3e458b5ce9a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:04.467929 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01334a54-c95a-4246-a67c-3e458b5ce9a9 None admin] PATH_INFO: `/v3/projects/f8cf6274da2e456eb20ee0ba0d187a83` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:04.469808 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01334a54-c95a-4246-a67c-3e458b5ce9a9 None admin] RBAC: Authorizing `identity:get_project(project_id=f8cf6274da2e456eb20ee0ba0d187a83)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:04.471295 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01334a54-c95a-4246-a67c-3e458b5ce9a9 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:04.473098 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 75/158] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Sep 9 11:59:04 2022] GET /identity/v3/projects/f8cf6274da2e456eb20ee0ba0d187a83 => generated 298 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:59:04.479482 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01334a54-c95a-4246-a67c-3e458b5ce9a9 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:04.482079 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa8d844-2c1e-47ae-a3ca-aca37ead3b99 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:04.556585 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5196996b-aeff-4bc3-b348-f1ac55f650af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:04.557276 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5196996b-aeff-4bc3-b348-f1ac55f650af None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:04.557689 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5196996b-aeff-4bc3-b348-f1ac55f650af None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:04.558067 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5196996b-aeff-4bc3-b348-f1ac55f650af None admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:04.558737 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5196996b-aeff-4bc3-b348-f1ac55f650af None admin] RBAC: Adding query filter params (name=member) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:59:04.559060 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5196996b-aeff-4bc3-b348-f1ac55f650af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:04.560458 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5196996b-aeff-4bc3-b348-f1ac55f650af None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:04.578938 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 84/159] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Sep 9 11:59:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:04.585390 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5196996b-aeff-4bc3-b348-f1ac55f650af None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:04.638935 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01334a54-c95a-4246-a67c-3e458b5ce9a9 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:04.707225 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172a15f8-cbc3-4866-96e4-6f0e9f9c4267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:04.707914 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172a15f8-cbc3-4866-96e4-6f0e9f9c4267 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:04.708186 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172a15f8-cbc3-4866-96e4-6f0e9f9c4267 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:04.709058 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172a15f8-cbc3-4866-96e4-6f0e9f9c4267 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:04.710157 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172a15f8-cbc3-4866-96e4-6f0e9f9c4267 None admin] RBAC: Adding query filter params (role.id=3f2ca33902a04b6dbc2ef7d06d7289d3, scope.project.id=f8cf6274da2e456eb20ee0ba0d187a83, user.id=136011715ec34418adc0f536f6c7357c) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:59:04.710460 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172a15f8-cbc3-4866-96e4-6f0e9f9c4267 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:04.711954 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172a15f8-cbc3-4866-96e4-6f0e9f9c4267 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:04.727870 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 76/160] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Sep 9 11:59:04 2022] GET /identity/v3/role_assignments?user.id=136011715ec34418adc0f536f6c7357c&scope.project.id=f8cf6274da2e456eb20ee0ba0d187a83&role.id=3f2ca33902a04b6dbc2ef7d06d7289d3 => generated 608 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:59:04.729126 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fe90ec-d916-40ce-9e6b-13049ded5b76 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:04.729432 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fe90ec-d916-40ce-9e6b-13049ded5b76 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:04.729707 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fe90ec-d916-40ce-9e6b-13049ded5b76 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:04.731236 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 77/161] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 11:59:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:59:04.740837 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c48e40-04cb-43e9-848b-f7cfefc50fab None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:04.741260 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c48e40-04cb-43e9-848b-f7cfefc50fab None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:04.741538 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c48e40-04cb-43e9-848b-f7cfefc50fab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:04.770621 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5196996b-aeff-4bc3-b348-f1ac55f650af None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:04.779964 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:04.781431 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-f4c48e40-04cb-43e9-848b-f7cfefc50fab None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:59:04.783369 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-f4c48e40-04cb-43e9-848b-f7cfefc50fab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:04.798043 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:04.837539 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ee0f00-d56e-40e2-a9a1-4ac2ff53c6ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:04.838105 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ee0f00-d56e-40e2-a9a1-4ac2ff53c6ba None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:04.838386 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ee0f00-d56e-40e2-a9a1-4ac2ff53c6ba None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:04.838660 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ee0f00-d56e-40e2-a9a1-4ac2ff53c6ba None admin] PATH_INFO: `/v3/users/1ccfba61dc7d4ad98da496e5800a8589` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:04.840765 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ee0f00-d56e-40e2-a9a1-4ac2ff53c6ba None admin] RBAC: Authorizing `identity:get_user(user_id=1ccfba61dc7d4ad98da496e5800a8589)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:04.843224 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ee0f00-d56e-40e2-a9a1-4ac2ff53c6ba None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:04.845104 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 85/162] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Sep 9 11:59:04 2022] GET /identity/v3/users/1ccfba61dc7d4ad98da496e5800a8589 => generated 275 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:59:04.852473 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ee0f00-d56e-40e2-a9a1-4ac2ff53c6ba None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:04.922700 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 78/163] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 11:59:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 11:59:05.012105 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ee0f00-d56e-40e2-a9a1-4ac2ff53c6ba None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:05.054272 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5015dc07-c33d-4384-bd57-539a6261165c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:05.054888 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5015dc07-c33d-4384-bd57-539a6261165c None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:05.055188 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5015dc07-c33d-4384-bd57-539a6261165c None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:05.055464 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5015dc07-c33d-4384-bd57-539a6261165c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:05.111576 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:05.132727 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736ef92e-f900-47b0-a62f-c578b7c0a65f None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:05.133071 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736ef92e-f900-47b0-a62f-c578b7c0a65f None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:05.133346 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736ef92e-f900-47b0-a62f-c578b7c0a65f None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:05.134922 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 79/164] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 11:59:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 11:59:05.140700 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d73fbf-9dc2-4a90-964b-9d52407b5c31 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:05.141303 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d73fbf-9dc2-4a90-964b-9d52407b5c31 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:05.141779 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d73fbf-9dc2-4a90-964b-9d52407b5c31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:05.148085 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5015dc07-c33d-4384-bd57-539a6261165c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:05.149724 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5015dc07-c33d-4384-bd57-539a6261165c None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:05.163329 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-91d73fbf-9dc2-4a90-964b-9d52407b5c31 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:59:05.164879 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-91d73fbf-9dc2-4a90-964b-9d52407b5c31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:05.283095 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 80/165] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 11:59:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 11:59:05.307302 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 86/166] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 9 11:59:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 11:59:05.312950 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d73fbf-9dc2-4a90-964b-9d52407b5c31 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:05.404869 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:05.437226 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d73fbf-9dc2-4a90-964b-9d52407b5c31 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:05.492082 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab94443e-5167-4aaf-b6d9-e6c04beae204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:05.492725 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab94443e-5167-4aaf-b6d9-e6c04beae204 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:05.493000 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab94443e-5167-4aaf-b6d9-e6c04beae204 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:05.493268 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab94443e-5167-4aaf-b6d9-e6c04beae204 None admin] PATH_INFO: `/v3/projects/f8cf6274da2e456eb20ee0ba0d187a83` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:05.494386 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab94443e-5167-4aaf-b6d9-e6c04beae204 None admin] RBAC: Authorizing `identity:get_project(project_id=f8cf6274da2e456eb20ee0ba0d187a83)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:05.495853 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab94443e-5167-4aaf-b6d9-e6c04beae204 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:05.497701 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 81/167] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Sep 9 11:59:05 2022] GET /identity/v3/projects/f8cf6274da2e456eb20ee0ba0d187a83 => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:05.503351 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5015dc07-c33d-4384-bd57-539a6261165c None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:05.606178 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:05.634948 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5015dc07-c33d-4384-bd57-539a6261165c None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:05.673208 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe947f7b-6dcf-407b-881b-0dbc852b2eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:05.673741 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe947f7b-6dcf-407b-881b-0dbc852b2eff None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:05.674020 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe947f7b-6dcf-407b-881b-0dbc852b2eff None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:05.674295 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe947f7b-6dcf-407b-881b-0dbc852b2eff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:05.675057 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe947f7b-6dcf-407b-881b-0dbc852b2eff None admin] RBAC: Adding query filter params (role.id=eecd2f160dd844f5906f3d345780c501, scope.project.id=f8cf6274da2e456eb20ee0ba0d187a83, user.id=1ccfba61dc7d4ad98da496e5800a8589) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:59:05.675365 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe947f7b-6dcf-407b-881b-0dbc852b2eff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:05.676830 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe947f7b-6dcf-407b-881b-0dbc852b2eff None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:05.684094 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 87/168] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Sep 9 11:59:05 2022] GET /identity/v3/role_assignments?user.id=1ccfba61dc7d4ad98da496e5800a8589&scope.project.id=f8cf6274da2e456eb20ee0ba0d187a83&role.id=eecd2f160dd844f5906f3d345780c501 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:05.701064 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab94443e-5167-4aaf-b6d9-e6c04beae204 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:05.811385 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe947f7b-6dcf-407b-881b-0dbc852b2eff None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:05.889187 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab94443e-5167-4aaf-b6d9-e6c04beae204 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:05.975511 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d52880-d955-4759-a8e7-9d3d2899126b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:05.976077 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d52880-d955-4759-a8e7-9d3d2899126b None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:05.976356 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d52880-d955-4759-a8e7-9d3d2899126b None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:05.976616 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d52880-d955-4759-a8e7-9d3d2899126b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:05.999878 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d52880-d955-4759-a8e7-9d3d2899126b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:59:06.004786 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-15d52880-d955-4759-a8e7-9d3d2899126b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 11:59:06.004786 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:59:06.004786 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:59:06.004786 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:59:06.004786 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:59:06.004786 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:59:06.004786 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:59:06.004786 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:59:06.004786 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:59:06.004786 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:59:06.004786 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:59:06.004786 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:59:06.004786 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:59:06.004786 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:59:06.004786 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:59:06.004786 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:59:06.006042 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:59:06.006042 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:59:06.006042 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:59:06.006042 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:59:06.006042 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:59:06.006042 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:59:06.006042 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:59:06.006042 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:59:06.006042 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:59:06.006042 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:59:06.006042 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:59:06.006042 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 11:59:06.006042 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:59:06.006042 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:59:06.006042 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:59:06.006042 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:59:06.006942 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:59:06.006942 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:59:06.006942 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:59:06.006942 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:59:06.006942 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:59:06.006942 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:59:06.006942 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:59:06.006942 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:59:06.006942 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:59:06.006942 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:59:06.006942 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 11:59:06.006942 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 11:59:06.010814 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 82/169] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 9 11:59:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:59:06.015394 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d52880-d955-4759-a8e7-9d3d2899126b None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:06.018910 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:06.041443 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe947f7b-6dcf-407b-881b-0dbc852b2eff None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:06.098496 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb483f0-63b9-47e9-80f4-6fd7af63c05d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:06.099249 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb483f0-63b9-47e9-80f4-6fd7af63c05d None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:06.099692 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb483f0-63b9-47e9-80f4-6fd7af63c05d None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:06.099973 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb483f0-63b9-47e9-80f4-6fd7af63c05d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:06.110865 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb483f0-63b9-47e9-80f4-6fd7af63c05d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:59:06.116427 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-bcb483f0-63b9-47e9-80f4-6fd7af63c05d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 11:59:06.116427 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:59:06.116427 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:59:06.116427 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:59:06.116427 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:59:06.116427 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:59:06.116427 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:59:06.116427 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:59:06.116427 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:59:06.116427 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:59:06.116427 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:59:06.116427 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:59:06.116427 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:59:06.116427 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:59:06.116427 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:59:06.116427 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:59:06.118509 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:59:06.118509 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:59:06.118509 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:59:06.118509 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:59:06.118509 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:59:06.118509 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:59:06.118509 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:59:06.118509 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:59:06.118509 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:59:06.118509 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:59:06.118509 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:59:06.118509 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 11:59:06.118509 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:59:06.118509 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:59:06.118509 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:59:06.118509 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:59:06.120317 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:59:06.120317 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:59:06.120317 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:59:06.120317 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:59:06.120317 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:59:06.120317 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:59:06.120317 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:59:06.120317 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:59:06.120317 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:59:06.120317 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:59:06.120317 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 11:59:06.120317 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 11:59:06.120317 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:06.120317 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 88/170] 10.222.0.49 () {56 vars in 1114 bytes} [Fri Sep 9 11:59:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:59:06.127156 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb483f0-63b9-47e9-80f4-6fd7af63c05d None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:06.264944 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d52880-d955-4759-a8e7-9d3d2899126b None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:06.287181 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb483f0-63b9-47e9-80f4-6fd7af63c05d None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:06.307193 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f64ede-9936-421b-8180-19d4db2ab68a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:06.307818 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f64ede-9936-421b-8180-19d4db2ab68a None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:06.308148 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f64ede-9936-421b-8180-19d4db2ab68a None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:06.308427 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f64ede-9936-421b-8180-19d4db2ab68a None admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:06.309109 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f64ede-9936-421b-8180-19d4db2ab68a None admin] RBAC: Adding query filter params (name=member) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:59:06.309405 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f64ede-9936-421b-8180-19d4db2ab68a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:06.310797 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f64ede-9936-421b-8180-19d4db2ab68a None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:06.320106 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 83/171] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Sep 9 11:59:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:59:06.327110 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f64ede-9936-421b-8180-19d4db2ab68a None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:06.366777 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715ebcc2-1779-43f9-9328-369a805f4f3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:06.367421 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715ebcc2-1779-43f9-9328-369a805f4f3e None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:06.367694 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715ebcc2-1779-43f9-9328-369a805f4f3e None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:06.367963 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715ebcc2-1779-43f9-9328-369a805f4f3e None admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:06.368626 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715ebcc2-1779-43f9-9328-369a805f4f3e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:59:06.368921 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715ebcc2-1779-43f9-9328-369a805f4f3e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:06.370311 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715ebcc2-1779-43f9-9328-369a805f4f3e None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:06.388611 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 89/172] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 9 11:59:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:06.395709 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715ebcc2-1779-43f9-9328-369a805f4f3e None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:06.471590 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:06.487273 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:06.537735 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715ebcc2-1779-43f9-9328-369a805f4f3e None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:06.556539 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f64ede-9936-421b-8180-19d4db2ab68a None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:06.607875 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283fc746-473d-4e76-bce7-1052d2ec60a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:06.608522 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283fc746-473d-4e76-bce7-1052d2ec60a0 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:06.608801 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283fc746-473d-4e76-bce7-1052d2ec60a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:06.609072 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283fc746-473d-4e76-bce7-1052d2ec60a0 None admin] PATH_INFO: `/v3/users/1ccfba61dc7d4ad98da496e5800a8589` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:06.611239 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283fc746-473d-4e76-bce7-1052d2ec60a0 None admin] RBAC: Authorizing `identity:get_user(user_id=1ccfba61dc7d4ad98da496e5800a8589)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:06.612692 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283fc746-473d-4e76-bce7-1052d2ec60a0 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:06.616744 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 90/173] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Sep 9 11:59:06 2022] GET /identity/v3/users/1ccfba61dc7d4ad98da496e5800a8589 => generated 275 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:06.621669 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283fc746-473d-4e76-bce7-1052d2ec60a0 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:06.634969 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3fb670-94d1-45bd-aecb-3e90ab8bdc09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:06.634969 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3fb670-94d1-45bd-aecb-3e90ab8bdc09 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:06.634969 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3fb670-94d1-45bd-aecb-3e90ab8bdc09 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:06.634969 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3fb670-94d1-45bd-aecb-3e90ab8bdc09 None admin] PATH_INFO: `/v3/users/1ccfba61dc7d4ad98da496e5800a8589` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:06.635363 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3fb670-94d1-45bd-aecb-3e90ab8bdc09 None admin] RBAC: Authorizing `identity:get_user(user_id=1ccfba61dc7d4ad98da496e5800a8589)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:06.636655 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3fb670-94d1-45bd-aecb-3e90ab8bdc09 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:06.638271 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 84/174] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Sep 9 11:59:06 2022] GET /identity/v3/users/1ccfba61dc7d4ad98da496e5800a8589 => generated 275 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:59:06.644264 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3fb670-94d1-45bd-aecb-3e90ab8bdc09 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:06.778681 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283fc746-473d-4e76-bce7-1052d2ec60a0 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:06.815606 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b651c125-933f-4af2-a2cb-8dd93da9a024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:06.816574 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b651c125-933f-4af2-a2cb-8dd93da9a024 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:06.816867 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b651c125-933f-4af2-a2cb-8dd93da9a024 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:06.817146 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b651c125-933f-4af2-a2cb-8dd93da9a024 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:06.831333 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:06.891311 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b651c125-933f-4af2-a2cb-8dd93da9a024 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:06.892829 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b651c125-933f-4af2-a2cb-8dd93da9a024 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:06.898065 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3fb670-94d1-45bd-aecb-3e90ab8bdc09 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:06.977942 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:07.002364 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26a1c72-6560-4c0c-89b4-2bda087ab1de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:07.003299 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26a1c72-6560-4c0c-89b4-2bda087ab1de None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:07.003738 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26a1c72-6560-4c0c-89b4-2bda087ab1de None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:07.004149 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26a1c72-6560-4c0c-89b4-2bda087ab1de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:07.010921 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 91/175] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 9 11:59:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 09 11:59:07.015403 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b651c125-933f-4af2-a2cb-8dd93da9a024 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:07.056871 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:07.117531 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26a1c72-6560-4c0c-89b4-2bda087ab1de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:07.119486 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26a1c72-6560-4c0c-89b4-2bda087ab1de None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:07.172210 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b651c125-933f-4af2-a2cb-8dd93da9a024 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:07.211433 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d97ed8-adf0-4993-9fc0-97bcda469393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:07.212073 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d97ed8-adf0-4993-9fc0-97bcda469393 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:07.212474 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d97ed8-adf0-4993-9fc0-97bcda469393 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:07.212827 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d97ed8-adf0-4993-9fc0-97bcda469393 None admin] PATH_INFO: `/v3/projects/f8cf6274da2e456eb20ee0ba0d187a83` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:07.214964 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d97ed8-adf0-4993-9fc0-97bcda469393 None admin] RBAC: Authorizing `identity:get_project(project_id=f8cf6274da2e456eb20ee0ba0d187a83)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:07.216431 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d97ed8-adf0-4993-9fc0-97bcda469393 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:07.218389 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 92/176] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Sep 9 11:59:07 2022] GET /identity/v3/projects/f8cf6274da2e456eb20ee0ba0d187a83 => generated 298 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:07.223892 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5676920f-7cc2-4d70-983b-ea2db510e324 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:07.224353 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5676920f-7cc2-4d70-983b-ea2db510e324 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:07.224662 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5676920f-7cc2-4d70-983b-ea2db510e324 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:07.225987 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 93/177] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 11:59:07 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:59:07.227615 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5c80c7-354f-43f7-b666-c9a701e2f5c7 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:07.228240 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5c80c7-354f-43f7-b666-c9a701e2f5c7 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:07.228515 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5c80c7-354f-43f7-b666-c9a701e2f5c7 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:07.230023 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 94/178] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 11:59:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:59:07.234664 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1042f1b0-3332-438c-bc21-45131de63dab None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:07.234664 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1042f1b0-3332-438c-bc21-45131de63dab None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:07.234664 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1042f1b0-3332-438c-bc21-45131de63dab None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:07.234664 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 95/179] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 11:59:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:59:07.234664 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1042f1b0-3332-438c-bc21-45131de63dab None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:07.260792 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 85/180] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 9 11:59:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 617 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 09 11:59:07.262085 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23962100-899f-4588-a3c5-4502dc3ba21a None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:07.262376 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23962100-899f-4588-a3c5-4502dc3ba21a None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:07.262647 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23962100-899f-4588-a3c5-4502dc3ba21a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:07.281816 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-23962100-899f-4588-a3c5-4502dc3ba21a None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:59:07.283017 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-23962100-899f-4588-a3c5-4502dc3ba21a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:07.310595 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:07.339025 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:07.348219 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1042f1b0-3332-438c-bc21-45131de63dab None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:07.363439 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 86/181] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 11:59:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:59:07.365802 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c13119-878a-4d87-b2d4-528c2264acec None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:07.366289 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c13119-878a-4d87-b2d4-528c2264acec None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:07.367056 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c13119-878a-4d87-b2d4-528c2264acec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:07.391940 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-47c13119-878a-4d87-b2d4-528c2264acec None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:59:07.393367 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98340e84-b4a9-42d3-9530-fa70077d214b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:07.393709 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-47c13119-878a-4d87-b2d4-528c2264acec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:07.394198 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98340e84-b4a9-42d3-9530-fa70077d214b None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:07.394565 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98340e84-b4a9-42d3-9530-fa70077d214b None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:07.394987 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98340e84-b4a9-42d3-9530-fa70077d214b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:07.395892 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98340e84-b4a9-42d3-9530-fa70077d214b None admin] RBAC: Adding query filter params (role.id=a7c71d2d9914408bb9953f85329d1e80, scope.project.id=f8cf6274da2e456eb20ee0ba0d187a83, user.id=1ccfba61dc7d4ad98da496e5800a8589) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:59:07.396432 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98340e84-b4a9-42d3-9530-fa70077d214b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:07.399347 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98340e84-b4a9-42d3-9530-fa70077d214b None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:07.406850 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 96/182] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Sep 9 11:59:07 2022] GET /identity/v3/role_assignments?user.id=1ccfba61dc7d4ad98da496e5800a8589&scope.project.id=f8cf6274da2e456eb20ee0ba0d187a83&role.id=a7c71d2d9914408bb9953f85329d1e80 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:07.408606 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef5d3ea-b386-408d-8033-663da2125c4a None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:07.409196 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef5d3ea-b386-408d-8033-663da2125c4a None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:07.409640 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef5d3ea-b386-408d-8033-663da2125c4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:07.435729 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-4ef5d3ea-b386-408d-8033-663da2125c4a None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:59:07.437068 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-4ef5d3ea-b386-408d-8033-663da2125c4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:07.491388 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 87/183] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 11:59:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:59:07.492859 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c13119-878a-4d87-b2d4-528c2264acec None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:07.569800 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 97/184] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 11:59:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:59:07.575251 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6486874d-b63c-4681-a456-d5eca0bdba1f None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:07.575853 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6486874d-b63c-4681-a456-d5eca0bdba1f None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:07.577941 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6486874d-b63c-4681-a456-d5eca0bdba1f None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:07.579874 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 98/185] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 11:59:07 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:59:07.585825 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdf8c09-4627-4ffd-9847-55f161c7251e None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:07.586428 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdf8c09-4627-4ffd-9847-55f161c7251e None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:07.586951 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdf8c09-4627-4ffd-9847-55f161c7251e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:07.607765 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-dfdf8c09-4627-4ffd-9847-55f161c7251e None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:59:07.609483 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-dfdf8c09-4627-4ffd-9847-55f161c7251e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:07.667350 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:07.677711 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c13119-878a-4d87-b2d4-528c2264acec None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:07.685114 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 99/186] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 11:59:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 11:59:07.685875 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdf8c09-4627-4ffd-9847-55f161c7251e None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:07.715594 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a03144-da80-4288-afcf-8f70c76fef63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:07.716231 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a03144-da80-4288-afcf-8f70c76fef63 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:07.716503 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a03144-da80-4288-afcf-8f70c76fef63 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:07.716771 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a03144-da80-4288-afcf-8f70c76fef63 None admin] PATH_INFO: `/v3/projects/2d1d07ce59064a289d3ee8acb5552db9` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:07.718566 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a03144-da80-4288-afcf-8f70c76fef63 None admin] RBAC: Authorizing `identity:get_project(project_id=2d1d07ce59064a289d3ee8acb5552db9)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:07.720177 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a03144-da80-4288-afcf-8f70c76fef63 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:07.721822 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 88/187] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Sep 9 11:59:07 2022] GET /identity/v3/projects/2d1d07ce59064a289d3ee8acb5552db9 => generated 312 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:07.728032 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:07.729686 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a03144-da80-4288-afcf-8f70c76fef63 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:07.855363 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a03144-da80-4288-afcf-8f70c76fef63 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:07.877436 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdf8c09-4627-4ffd-9847-55f161c7251e None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:07.924448 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de1bea3-75b6-4f1b-9c30-9e41b4e80dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:07.925003 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de1bea3-75b6-4f1b-9c30-9e41b4e80dc1 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:07.925228 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de1bea3-75b6-4f1b-9c30-9e41b4e80dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:07.925506 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de1bea3-75b6-4f1b-9c30-9e41b4e80dc1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:07.926236 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de1bea3-75b6-4f1b-9c30-9e41b4e80dc1 None admin] RBAC: Adding query filter params (role.id=eecd2f160dd844f5906f3d345780c501, scope.project.id=2d1d07ce59064a289d3ee8acb5552db9, user.id=1ccfba61dc7d4ad98da496e5800a8589) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:59:07.926544 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de1bea3-75b6-4f1b-9c30-9e41b4e80dc1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:07.937055 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de1bea3-75b6-4f1b-9c30-9e41b4e80dc1 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:07.944985 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 89/188] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Sep 9 11:59:07 2022] GET /identity/v3/role_assignments?user.id=1ccfba61dc7d4ad98da496e5800a8589&scope.project.id=2d1d07ce59064a289d3ee8acb5552db9&role.id=eecd2f160dd844f5906f3d345780c501 => generated 608 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:59:07.948167 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de1bea3-75b6-4f1b-9c30-9e41b4e80dc1 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:07.953866 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d85b27-ff63-41af-a04e-1c158a40a927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:07.954347 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d85b27-ff63-41af-a04e-1c158a40a927 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:07.954626 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d85b27-ff63-41af-a04e-1c158a40a927 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:07.954911 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d85b27-ff63-41af-a04e-1c158a40a927 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:07.980036 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:07.980867 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d85b27-ff63-41af-a04e-1c158a40a927 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:59:07.986461 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-25d85b27-ff63-41af-a04e-1c158a40a927 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 11:59:07.986461 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:59:07.986461 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:59:07.986461 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:59:07.986461 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:59:07.986461 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:59:07.986461 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:59:07.986461 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:59:07.986461 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:59:07.986461 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:59:07.986461 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:59:07.986461 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:59:07.986461 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:59:07.986461 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:59:07.986461 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:59:07.986461 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:59:07.987419 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:59:07.987419 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:59:07.987419 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:59:07.987419 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:59:07.987419 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:59:07.987419 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:59:07.987419 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:59:07.987419 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:59:07.987419 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:59:07.987419 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:59:07.987419 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:59:07.987419 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 11:59:07.987419 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:59:07.987419 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:59:07.987419 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:59:07.987419 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:59:07.988195 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:59:07.988195 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:59:07.988195 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:59:07.988195 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:59:07.988195 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:59:07.988195 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:59:07.988195 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:59:07.988195 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:59:07.988195 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:59:07.988195 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:59:07.988195 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 11:59:07.988195 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 11:59:07.988195 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 100/189] 10.222.0.49 () {56 vars in 1114 bytes} [Fri Sep 9 11:59:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:59:07.995247 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d85b27-ff63-41af-a04e-1c158a40a927 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:08.073622 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de1bea3-75b6-4f1b-9c30-9e41b4e80dc1 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:08.103156 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:08.110854 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f7798c-e3ef-47e3-b2ce-af133b89725a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:08.111535 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f7798c-e3ef-47e3-b2ce-af133b89725a None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:08.111815 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f7798c-e3ef-47e3-b2ce-af133b89725a None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:08.112093 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f7798c-e3ef-47e3-b2ce-af133b89725a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:08.125282 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f7798c-e3ef-47e3-b2ce-af133b89725a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:59:08.125918 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d85b27-ff63-41af-a04e-1c158a40a927 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:08.130369 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-21f7798c-e3ef-47e3-b2ce-af133b89725a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:59:08.130369 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:59:08.130369 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:59:08.130369 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:59:08.130369 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:59:08.130369 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:59:08.130369 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:59:08.130369 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:59:08.130369 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:59:08.130369 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:59:08.130369 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:59:08.130369 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:59:08.130369 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:59:08.130369 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:59:08.130369 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:59:08.130369 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:59:08.131816 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:59:08.131816 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:59:08.131816 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:59:08.131816 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:59:08.131816 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:59:08.131816 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:59:08.131816 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:59:08.131816 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:59:08.131816 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:59:08.131816 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:59:08.131816 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:59:08.131816 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 11:59:08.131816 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:59:08.131816 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:59:08.131816 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:59:08.131816 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:59:08.132629 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:59:08.132629 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:59:08.132629 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:59:08.132629 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:59:08.132629 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:59:08.132629 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:59:08.132629 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:59:08.132629 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:59:08.132629 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:59:08.132629 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:59:08.132629 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:59:08.132629 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 11:59:08.137159 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 90/190] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Sep 9 11:59:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:59:08.138000 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f7798c-e3ef-47e3-b2ce-af133b89725a None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:08.166030 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0b0246-f96c-4920-b585-2700afa989c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:08.166665 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0b0246-f96c-4920-b585-2700afa989c4 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:08.166963 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0b0246-f96c-4920-b585-2700afa989c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:08.167229 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0b0246-f96c-4920-b585-2700afa989c4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:08.169348 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0b0246-f96c-4920-b585-2700afa989c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:08.170968 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0b0246-f96c-4920-b585-2700afa989c4 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:08.172137 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 101/191] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 11:59:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:08.172889 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0b0246-f96c-4920-b585-2700afa989c4 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:08.240765 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:08.257184 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f7798c-e3ef-47e3-b2ce-af133b89725a None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:08.288124 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0b0246-f96c-4920-b585-2700afa989c4 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:08.291444 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aca94a2-1354-4395-85d9-7c357a0a4b86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:08.292050 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aca94a2-1354-4395-85d9-7c357a0a4b86 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:08.292327 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aca94a2-1354-4395-85d9-7c357a0a4b86 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:08.292600 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aca94a2-1354-4395-85d9-7c357a0a4b86 None admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:08.293261 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aca94a2-1354-4395-85d9-7c357a0a4b86 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:59:08.293563 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aca94a2-1354-4395-85d9-7c357a0a4b86 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:08.294974 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aca94a2-1354-4395-85d9-7c357a0a4b86 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:08.304001 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 91/192] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 9 11:59:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:08.304804 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aca94a2-1354-4395-85d9-7c357a0a4b86 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:08.326395 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be85137c-49d8-48e7-ba4b-7146738bffc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:08.327058 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be85137c-49d8-48e7-ba4b-7146738bffc3 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:08.327338 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be85137c-49d8-48e7-ba4b-7146738bffc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:08.327609 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be85137c-49d8-48e7-ba4b-7146738bffc3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:08.338104 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be85137c-49d8-48e7-ba4b-7146738bffc3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:59:08.342923 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-be85137c-49d8-48e7-ba4b-7146738bffc3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 11:59:08.342923 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:59:08.342923 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:59:08.342923 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:59:08.342923 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:59:08.342923 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:59:08.342923 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:59:08.342923 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:59:08.342923 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:59:08.342923 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:59:08.342923 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:59:08.342923 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:59:08.342923 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:59:08.342923 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:59:08.342923 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:59:08.342923 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:59:08.343815 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:59:08.343815 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:59:08.343815 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:59:08.343815 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:59:08.343815 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:59:08.343815 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:59:08.343815 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:59:08.343815 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:59:08.343815 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:59:08.343815 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:59:08.343815 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:59:08.343815 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 11:59:08.343815 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:59:08.343815 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:59:08.343815 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:59:08.343815 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:59:08.344606 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:59:08.344606 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:59:08.344606 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:59:08.344606 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:59:08.344606 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:59:08.344606 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:59:08.344606 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:59:08.344606 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:59:08.344606 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:59:08.344606 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:59:08.344606 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 11:59:08.344606 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 11:59:08.344606 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 102/193] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 9 11:59:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:59:08.345284 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be85137c-49d8-48e7-ba4b-7146738bffc3 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:08.357202 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:08.407694 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aca94a2-1354-4395-85d9-7c357a0a4b86 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:08.440535 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe9cdd8-a40f-422c-8a6b-d235826f83ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:08.441139 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe9cdd8-a40f-422c-8a6b-d235826f83ea None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:08.441418 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe9cdd8-a40f-422c-8a6b-d235826f83ea None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:08.441792 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe9cdd8-a40f-422c-8a6b-d235826f83ea None admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:08.442350 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe9cdd8-a40f-422c-8a6b-d235826f83ea None admin] RBAC: Adding query filter params (name=admin) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:59:08.442655 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe9cdd8-a40f-422c-8a6b-d235826f83ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:08.444591 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe9cdd8-a40f-422c-8a6b-d235826f83ea None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:08.445881 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be85137c-49d8-48e7-ba4b-7146738bffc3 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:08.452832 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 92/194] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 9 11:59:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:08.453407 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe9cdd8-a40f-422c-8a6b-d235826f83ea None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:08.464544 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:08.480215 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66184ec1-7544-475a-a4ce-c06ff1fcc3aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:08.480817 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66184ec1-7544-475a-a4ce-c06ff1fcc3aa None admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:08.481094 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66184ec1-7544-475a-a4ce-c06ff1fcc3aa None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:08.481362 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66184ec1-7544-475a-a4ce-c06ff1fcc3aa None admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:08.482007 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66184ec1-7544-475a-a4ce-c06ff1fcc3aa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:08.483474 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66184ec1-7544-475a-a4ce-c06ff1fcc3aa None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:08.504117 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 103/195] 10.222.0.49 () {60 vars in 1132 bytes} [Fri Sep 9 11:59:08 2022] POST /identity/v3/users => generated 297 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:59:08.504932 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66184ec1-7544-475a-a4ce-c06ff1fcc3aa None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:08.563462 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe9cdd8-a40f-422c-8a6b-d235826f83ea None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:08.582078 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:08.604881 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b81d846-1725-40dd-8b24-a5cdab70069d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:08.605926 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b81d846-1725-40dd-8b24-a5cdab70069d None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:08.606399 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b81d846-1725-40dd-8b24-a5cdab70069d None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:08.606871 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b81d846-1725-40dd-8b24-a5cdab70069d None admin] PATH_INFO: `/v3/users/d4c01e0644c64ba1931a3d9db6b16207` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:08.622112 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b81d846-1725-40dd-8b24-a5cdab70069d None admin] RBAC: Authorizing `identity:get_user(user_id=d4c01e0644c64ba1931a3d9db6b16207)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:08.623985 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b81d846-1725-40dd-8b24-a5cdab70069d None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:08.630947 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 93/196] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Sep 9 11:59:08 2022] GET /identity/v3/users/d4c01e0644c64ba1931a3d9db6b16207 => generated 283 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:08.630947 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b81d846-1725-40dd-8b24-a5cdab70069d None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:08.632320 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66184ec1-7544-475a-a4ce-c06ff1fcc3aa None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:08.673692 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92cd41d-bbfb-46f2-b202-aa581e77aaee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:08.674306 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92cd41d-bbfb-46f2-b202-aa581e77aaee None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:08.674579 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92cd41d-bbfb-46f2-b202-aa581e77aaee None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:08.674847 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92cd41d-bbfb-46f2-b202-aa581e77aaee None admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:08.675552 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92cd41d-bbfb-46f2-b202-aa581e77aaee None admin] RBAC: Adding query filter params (name=reader) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:59:08.675845 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92cd41d-bbfb-46f2-b202-aa581e77aaee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:08.677926 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92cd41d-bbfb-46f2-b202-aa581e77aaee None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:08.690061 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 104/197] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Sep 9 11:59:08 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:08.692197 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92cd41d-bbfb-46f2-b202-aa581e77aaee None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:08.719706 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:08.767280 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b81d846-1725-40dd-8b24-a5cdab70069d None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:08.813473 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1a13fc-e1b5-43a6-a352-7be9acd1a04b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:08.814379 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1a13fc-e1b5-43a6-a352-7be9acd1a04b None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:08.814653 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1a13fc-e1b5-43a6-a352-7be9acd1a04b None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:08.817718 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1a13fc-e1b5-43a6-a352-7be9acd1a04b None admin] PATH_INFO: `/v3/users/d4c01e0644c64ba1931a3d9db6b16207` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:08.817718 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1a13fc-e1b5-43a6-a352-7be9acd1a04b None admin] RBAC: Authorizing `identity:get_user(user_id=d4c01e0644c64ba1931a3d9db6b16207)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:08.818421 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92cd41d-bbfb-46f2-b202-aa581e77aaee None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:08.819442 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1a13fc-e1b5-43a6-a352-7be9acd1a04b None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:08.821066 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 94/198] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Sep 9 11:59:08 2022] GET /identity/v3/users/d4c01e0644c64ba1931a3d9db6b16207 => generated 283 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:08.821832 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1a13fc-e1b5-43a6-a352-7be9acd1a04b None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:08.854955 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:08.867833 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e642ea-d536-4909-8a74-ad9e8645b707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:08.867833 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e642ea-d536-4909-8a74-ad9e8645b707 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:08.868078 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e642ea-d536-4909-8a74-ad9e8645b707 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:08.868348 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e642ea-d536-4909-8a74-ad9e8645b707 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:08.937006 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1a13fc-e1b5-43a6-a352-7be9acd1a04b None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:08.961784 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e642ea-d536-4909-8a74-ad9e8645b707 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:08.963932 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e642ea-d536-4909-8a74-ad9e8645b707 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:08.987751 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733549b6-bd9e-402c-9763-ce2a6e31b112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:08.988388 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733549b6-bd9e-402c-9763-ce2a6e31b112 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:08.988660 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733549b6-bd9e-402c-9763-ce2a6e31b112 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:08.988924 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733549b6-bd9e-402c-9763-ce2a6e31b112 None admin] PATH_INFO: `/v3/users/4b07995ae7d54b759029ad81aa305946` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:08.991453 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733549b6-bd9e-402c-9763-ce2a6e31b112 None admin] RBAC: Authorizing `identity:get_user(user_id=4b07995ae7d54b759029ad81aa305946)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:08.992901 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733549b6-bd9e-402c-9763-ce2a6e31b112 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:08.994610 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 95/199] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Sep 9 11:59:08 2022] GET /identity/v3/users/4b07995ae7d54b759029ad81aa305946 => generated 289 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:08.995357 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733549b6-bd9e-402c-9763-ce2a6e31b112 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:08.998218 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:09.091127 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 105/200] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 9 11:59:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 11:59:09.091469 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e642ea-d536-4909-8a74-ad9e8645b707 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:09.119244 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733549b6-bd9e-402c-9763-ce2a6e31b112 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:09.126582 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:09.155571 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc29c31-a5b4-459e-9d2e-61c2de45e390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:09.156187 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc29c31-a5b4-459e-9d2e-61c2de45e390 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:09.156532 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc29c31-a5b4-459e-9d2e-61c2de45e390 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:09.156732 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc29c31-a5b4-459e-9d2e-61c2de45e390 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:09.221772 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e642ea-d536-4909-8a74-ad9e8645b707 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:09.239638 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc29c31-a5b4-459e-9d2e-61c2de45e390 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:09.242749 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc29c31-a5b4-459e-9d2e-61c2de45e390 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:09.244905 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:09.262582 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91e373e-7728-49a7-95e3-7758a45450c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:09.263312 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91e373e-7728-49a7-95e3-7758a45450c7 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:09.263678 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91e373e-7728-49a7-95e3-7758a45450c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:09.264226 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91e373e-7728-49a7-95e3-7758a45450c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:09.340583 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91e373e-7728-49a7-95e3-7758a45450c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:09.342288 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91e373e-7728-49a7-95e3-7758a45450c7 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:09.374463 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:09.382797 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 96/201] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 9 11:59:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 11:59:09.388146 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc29c31-a5b4-459e-9d2e-61c2de45e390 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:09.467471 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 106/202] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 9 11:59:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 11:59:09.468790 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91e373e-7728-49a7-95e3-7758a45450c7 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:09.506465 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc29c31-a5b4-459e-9d2e-61c2de45e390 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:09.508156 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:09.546126 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8a2959-49ec-44ff-a68f-30192fae7773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:09.547219 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8a2959-49ec-44ff-a68f-30192fae7773 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:09.547707 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8a2959-49ec-44ff-a68f-30192fae7773 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:09.548235 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8a2959-49ec-44ff-a68f-30192fae7773 None admin] PATH_INFO: `/v3/projects/5f8dc54c23f543d197444a43c6b82c01` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:09.549843 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8a2959-49ec-44ff-a68f-30192fae7773 None admin] RBAC: Authorizing `identity:get_project(project_id=5f8dc54c23f543d197444a43c6b82c01)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:09.551611 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8a2959-49ec-44ff-a68f-30192fae7773 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:09.554299 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 97/203] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Sep 9 11:59:09 2022] GET /identity/v3/projects/5f8dc54c23f543d197444a43c6b82c01 => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:09.559916 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8a2959-49ec-44ff-a68f-30192fae7773 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:09.597959 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91e373e-7728-49a7-95e3-7758a45450c7 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:09.637893 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:09.654937 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30320d33-6898-4c7b-8cef-1a0b041e3e9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:09.655840 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30320d33-6898-4c7b-8cef-1a0b041e3e9c None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:09.656334 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30320d33-6898-4c7b-8cef-1a0b041e3e9c None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:09.656820 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30320d33-6898-4c7b-8cef-1a0b041e3e9c None admin] PATH_INFO: `/v3/projects/5f8dc54c23f543d197444a43c6b82c01` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:09.658336 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30320d33-6898-4c7b-8cef-1a0b041e3e9c None admin] RBAC: Authorizing `identity:get_project(project_id=5f8dc54c23f543d197444a43c6b82c01)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:09.660150 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30320d33-6898-4c7b-8cef-1a0b041e3e9c None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:09.662131 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 107/204] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Sep 9 11:59:09 2022] GET /identity/v3/projects/5f8dc54c23f543d197444a43c6b82c01 => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:09.663129 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30320d33-6898-4c7b-8cef-1a0b041e3e9c None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:09.679203 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8a2959-49ec-44ff-a68f-30192fae7773 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:09.716066 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f71aaa-947d-479d-a89d-15578b94868a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:09.717049 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f71aaa-947d-479d-a89d-15578b94868a None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:09.717522 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f71aaa-947d-479d-a89d-15578b94868a None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:09.718000 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f71aaa-947d-479d-a89d-15578b94868a None admin] PATH_INFO: `/v3/projects/f8cf6274da2e456eb20ee0ba0d187a83` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:09.719466 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f71aaa-947d-479d-a89d-15578b94868a None admin] RBAC: Authorizing `identity:get_project(project_id=f8cf6274da2e456eb20ee0ba0d187a83)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:09.721231 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f71aaa-947d-479d-a89d-15578b94868a None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:09.723082 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 98/205] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Sep 9 11:59:09 2022] GET /identity/v3/projects/f8cf6274da2e456eb20ee0ba0d187a83 => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:09.724175 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f71aaa-947d-479d-a89d-15578b94868a None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:09.765311 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:09.792629 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30320d33-6898-4c7b-8cef-1a0b041e3e9c None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:09.830068 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f71aaa-947d-479d-a89d-15578b94868a None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:09.836116 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2175d4c-5d77-4cbc-99be-68a7db54fff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:09.836736 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2175d4c-5d77-4cbc-99be-68a7db54fff6 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:09.837013 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2175d4c-5d77-4cbc-99be-68a7db54fff6 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:09.837285 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2175d4c-5d77-4cbc-99be-68a7db54fff6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:09.838007 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2175d4c-5d77-4cbc-99be-68a7db54fff6 None admin] RBAC: Adding query filter params (role.id=a7c71d2d9914408bb9953f85329d1e80, scope.project.id=5f8dc54c23f543d197444a43c6b82c01, user.id=d4c01e0644c64ba1931a3d9db6b16207) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:59:09.838311 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2175d4c-5d77-4cbc-99be-68a7db54fff6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:09.840209 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2175d4c-5d77-4cbc-99be-68a7db54fff6 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:09.847137 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 108/206] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Sep 9 11:59:09 2022] GET /identity/v3/role_assignments?user.id=d4c01e0644c64ba1931a3d9db6b16207&scope.project.id=5f8dc54c23f543d197444a43c6b82c01&role.id=a7c71d2d9914408bb9953f85329d1e80 => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:09.853418 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2175d4c-5d77-4cbc-99be-68a7db54fff6 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:09.871309 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72ecfa3-48da-4211-b5a2-7ad5f3aa0eae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:09.871929 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72ecfa3-48da-4211-b5a2-7ad5f3aa0eae None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:09.872206 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72ecfa3-48da-4211-b5a2-7ad5f3aa0eae None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:09.872472 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72ecfa3-48da-4211-b5a2-7ad5f3aa0eae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:09.873211 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72ecfa3-48da-4211-b5a2-7ad5f3aa0eae None admin] RBAC: Adding query filter params (role.id=3f2ca33902a04b6dbc2ef7d06d7289d3, scope.project.id=5f8dc54c23f543d197444a43c6b82c01, user.id=d4c01e0644c64ba1931a3d9db6b16207) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:59:09.873515 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72ecfa3-48da-4211-b5a2-7ad5f3aa0eae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:09.876461 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72ecfa3-48da-4211-b5a2-7ad5f3aa0eae None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:09.891454 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 99/207] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Sep 9 11:59:09 2022] GET /identity/v3/role_assignments?user.id=d4c01e0644c64ba1931a3d9db6b16207&scope.project.id=5f8dc54c23f543d197444a43c6b82c01&role.id=3f2ca33902a04b6dbc2ef7d06d7289d3 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:09.991566 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:10.066306 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2175d4c-5d77-4cbc-99be-68a7db54fff6 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:10.112992 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150484d2-ae9c-49e1-b96d-1dfa2c325179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:10.113524 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150484d2-ae9c-49e1-b96d-1dfa2c325179 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:10.113805 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150484d2-ae9c-49e1-b96d-1dfa2c325179 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:10.114075 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150484d2-ae9c-49e1-b96d-1dfa2c325179 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:10.114858 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150484d2-ae9c-49e1-b96d-1dfa2c325179 None admin] RBAC: Adding query filter params (role.id=b1e7a1e6ddc54da7ba995fb55fd93016, scope.project.id=f8cf6274da2e456eb20ee0ba0d187a83, user.id=4b07995ae7d54b759029ad81aa305946) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:59:10.115414 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150484d2-ae9c-49e1-b96d-1dfa2c325179 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:10.116880 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150484d2-ae9c-49e1-b96d-1dfa2c325179 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:10.125208 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 109/208] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Sep 9 11:59:09 2022] GET /identity/v3/role_assignments?user.id=4b07995ae7d54b759029ad81aa305946&scope.project.id=f8cf6274da2e456eb20ee0ba0d187a83&role.id=b1e7a1e6ddc54da7ba995fb55fd93016 => generated 608 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:10.893973 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3202bd02-58d1-474b-b695-e466653904e1 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:10.894508 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3202bd02-58d1-474b-b695-e466653904e1 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:10.894508 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3202bd02-58d1-474b-b695-e466653904e1 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:10.896480 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 100/209] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 11:59:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:59:10.902158 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a90592-3953-4f4b-b4c5-50d015db5aad None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:10.902440 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a90592-3953-4f4b-b4c5-50d015db5aad None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:10.902708 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a90592-3953-4f4b-b4c5-50d015db5aad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:10.923931 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-a1a90592-3953-4f4b-b4c5-50d015db5aad None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:59:10.925115 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-a1a90592-3953-4f4b-b4c5-50d015db5aad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:11.016241 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 110/210] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 11:59:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:59:11.089600 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586b6ab4-4605-41ae-a65c-a4737c17d85b None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:11.089880 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586b6ab4-4605-41ae-a65c-a4737c17d85b None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:11.090153 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586b6ab4-4605-41ae-a65c-a4737c17d85b None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:11.091522 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 101/211] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 11:59:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:59:11.096817 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8643eb6-8ea8-463d-8f98-92fff05198d7 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:11.097101 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8643eb6-8ea8-463d-8f98-92fff05198d7 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:11.097371 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8643eb6-8ea8-463d-8f98-92fff05198d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:11.116662 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-a8643eb6-8ea8-463d-8f98-92fff05198d7 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:59:11.117832 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-a8643eb6-8ea8-463d-8f98-92fff05198d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:11.166915 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:11.191129 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 111/212] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 11:59:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:59:11.293821 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586b6ab4-4605-41ae-a65c-a4737c17d85b None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:11.400519 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586b6ab4-4605-41ae-a65c-a4737c17d85b None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:11.402068 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:11.435897 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8ff3c9-8db1-47ad-bbd4-6a0383c92838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:11.436448 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8ff3c9-8db1-47ad-bbd4-6a0383c92838 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:11.436727 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8ff3c9-8db1-47ad-bbd4-6a0383c92838 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:11.436996 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8ff3c9-8db1-47ad-bbd4-6a0383c92838 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:11.447657 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8ff3c9-8db1-47ad-bbd4-6a0383c92838 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:59:11.452443 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-5e8ff3c9-8db1-47ad-bbd4-6a0383c92838 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 11:59:11.452443 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:59:11.452443 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:59:11.452443 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:59:11.452443 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:59:11.452443 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:59:11.452443 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:59:11.452443 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:59:11.452443 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:59:11.452443 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:59:11.452443 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:59:11.452443 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:59:11.452443 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:59:11.452443 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:59:11.452443 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:59:11.452443 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:59:11.453309 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:59:11.453309 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:59:11.453309 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:59:11.453309 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:59:11.453309 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:59:11.453309 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:59:11.453309 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:59:11.453309 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:59:11.453309 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:59:11.453309 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:59:11.453309 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:59:11.453309 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 11:59:11.453309 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:59:11.453309 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:59:11.453309 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:59:11.453309 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:59:11.454136 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:59:11.454136 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:59:11.454136 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:59:11.454136 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:59:11.454136 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:59:11.454136 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:59:11.454136 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:59:11.454136 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:59:11.454136 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:59:11.454136 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:59:11.454136 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 11:59:11.454136 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 11:59:11.454136 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 102/213] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 9 11:59:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:59:11.460921 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8643eb6-8ea8-463d-8f98-92fff05198d7 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:11.469387 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8ff3c9-8db1-47ad-bbd4-6a0383c92838 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:11.570996 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8643eb6-8ea8-463d-8f98-92fff05198d7 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:11.575014 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8ff3c9-8db1-47ad-bbd4-6a0383c92838 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:11.607213 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335ae2ef-469a-4904-bc7d-f4081082b91d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:11.608218 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335ae2ef-469a-4904-bc7d-f4081082b91d None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:11.608567 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335ae2ef-469a-4904-bc7d-f4081082b91d None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:11.608838 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335ae2ef-469a-4904-bc7d-f4081082b91d None admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:11.609516 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335ae2ef-469a-4904-bc7d-f4081082b91d None admin] RBAC: Adding query filter params (name=member) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:59:11.609800 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335ae2ef-469a-4904-bc7d-f4081082b91d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:11.610991 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77181b59-3e64-4458-ba46-799853a28054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:11.611312 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335ae2ef-469a-4904-bc7d-f4081082b91d None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:11.612006 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77181b59-3e64-4458-ba46-799853a28054 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:11.612454 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77181b59-3e64-4458-ba46-799853a28054 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:11.612925 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77181b59-3e64-4458-ba46-799853a28054 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:11.614212 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77181b59-3e64-4458-ba46-799853a28054 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:11.615975 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77181b59-3e64-4458-ba46-799853a28054 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:11.618076 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 103/214] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 11:59:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:59:11.622939 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 112/215] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Sep 9 11:59:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:11.623983 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77181b59-3e64-4458-ba46-799853a28054 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:11.632267 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335ae2ef-469a-4904-bc7d-f4081082b91d None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:11.676000 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:11.696548 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:11.757249 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335ae2ef-469a-4904-bc7d-f4081082b91d None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:11.771992 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77181b59-3e64-4458-ba46-799853a28054 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:11.800175 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191af80c-1b18-4eee-a496-ae78f65784a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:11.801197 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191af80c-1b18-4eee-a496-ae78f65784a4 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:11.801965 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191af80c-1b18-4eee-a496-ae78f65784a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:11.802942 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191af80c-1b18-4eee-a496-ae78f65784a4 None admin] PATH_INFO: `/v3/users/976211872e2f4f188fa5e2eb48140809` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:11.811519 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d15931-8803-495e-8047-40e8504959b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:11.812308 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d15931-8803-495e-8047-40e8504959b2 None admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:11.812682 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d15931-8803-495e-8047-40e8504959b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:11.813046 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d15931-8803-495e-8047-40e8504959b2 None admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:11.813786 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d15931-8803-495e-8047-40e8504959b2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:11.815760 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d15931-8803-495e-8047-40e8504959b2 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:11.821240 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191af80c-1b18-4eee-a496-ae78f65784a4 None admin] RBAC: Authorizing `identity:get_user(user_id=976211872e2f4f188fa5e2eb48140809)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:11.822892 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191af80c-1b18-4eee-a496-ae78f65784a4 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:11.825303 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 113/216] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Sep 9 11:59:11 2022] GET /identity/v3/users/976211872e2f4f188fa5e2eb48140809 => generated 297 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:11.830938 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191af80c-1b18-4eee-a496-ae78f65784a4 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:11.842632 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 104/217] 10.222.0.49 () {60 vars in 1132 bytes} [Fri Sep 9 11:59:11 2022] POST /identity/v3/users => generated 297 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:59:11.955507 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191af80c-1b18-4eee-a496-ae78f65784a4 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:11.957097 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:11.998469 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e98dfdc-8b66-4deb-8580-e9a60947a0d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:11.999527 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e98dfdc-8b66-4deb-8580-e9a60947a0d9 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:12.000006 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e98dfdc-8b66-4deb-8580-e9a60947a0d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:12.000472 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e98dfdc-8b66-4deb-8580-e9a60947a0d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:12.086209 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e98dfdc-8b66-4deb-8580-e9a60947a0d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:12.087725 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e98dfdc-8b66-4deb-8580-e9a60947a0d9 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:12.206415 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 114/218] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 9 11:59:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 11:59:12.212423 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d15931-8803-495e-8047-40e8504959b2 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:12.283357 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59c4e8c-464e-4bba-860c-ffaa2cbc53ad None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:12.283655 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59c4e8c-464e-4bba-860c-ffaa2cbc53ad None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:12.283916 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59c4e8c-464e-4bba-860c-ffaa2cbc53ad None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:12.287063 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 115/219] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 11:59:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 11:59:12.291250 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e629f430-d61f-4a0d-a3e4-e569b59fab4f None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:12.291538 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e629f430-d61f-4a0d-a3e4-e569b59fab4f None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:12.291812 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e629f430-d61f-4a0d-a3e4-e569b59fab4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:12.300408 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:12.318339 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:12.321592 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-e629f430-d61f-4a0d-a3e4-e569b59fab4f None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:59:12.322443 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-e629f430-d61f-4a0d-a3e4-e569b59fab4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:12.339085 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d15931-8803-495e-8047-40e8504959b2 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:12.381821 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914ba63d-4d52-4f21-bc5d-8516ca2c3338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:12.382476 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914ba63d-4d52-4f21-bc5d-8516ca2c3338 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:12.382755 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914ba63d-4d52-4f21-bc5d-8516ca2c3338 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:12.383065 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914ba63d-4d52-4f21-bc5d-8516ca2c3338 None admin] PATH_INFO: `/v3/projects/5f8dc54c23f543d197444a43c6b82c01` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:12.385174 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914ba63d-4d52-4f21-bc5d-8516ca2c3338 None admin] RBAC: Authorizing `identity:get_project(project_id=5f8dc54c23f543d197444a43c6b82c01)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:12.386646 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914ba63d-4d52-4f21-bc5d-8516ca2c3338 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:12.388341 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 105/220] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Sep 9 11:59:12 2022] GET /identity/v3/projects/5f8dc54c23f543d197444a43c6b82c01 => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:12.389484 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3033d6-c442-46d3-9d5d-84f764662e72 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:12.389767 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3033d6-c442-46d3-9d5d-84f764662e72 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:12.390041 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3033d6-c442-46d3-9d5d-84f764662e72 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:12.396996 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 106/221] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 11:59:12 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:59:12.397823 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3033d6-c442-46d3-9d5d-84f764662e72 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:12.413796 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 116/222] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 11:59:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 11:59:12.415424 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad25454d-af45-4fb9-980b-4b47a2cd29d7 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:12.415710 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad25454d-af45-4fb9-980b-4b47a2cd29d7 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:12.415983 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad25454d-af45-4fb9-980b-4b47a2cd29d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:12.452172 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-ad25454d-af45-4fb9-980b-4b47a2cd29d7 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:59:12.455196 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-ad25454d-af45-4fb9-980b-4b47a2cd29d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:12.576686 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3033d6-c442-46d3-9d5d-84f764662e72 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:12.598414 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 117/223] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 11:59:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:59:12.652604 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bede0207-64a9-4582-b525-283eabc6935b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:12.653538 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bede0207-64a9-4582-b525-283eabc6935b None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:12.654017 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bede0207-64a9-4582-b525-283eabc6935b None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:12.654490 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bede0207-64a9-4582-b525-283eabc6935b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:12.655461 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bede0207-64a9-4582-b525-283eabc6935b None admin] RBAC: Adding query filter params (role.id=eecd2f160dd844f5906f3d345780c501, scope.project.id=5f8dc54c23f543d197444a43c6b82c01, user.id=976211872e2f4f188fa5e2eb48140809) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:59:12.656005 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bede0207-64a9-4582-b525-283eabc6935b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:12.657656 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bede0207-64a9-4582-b525-283eabc6935b None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:12.668009 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 107/224] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Sep 9 11:59:12 2022] GET /identity/v3/role_assignments?user.id=976211872e2f4f188fa5e2eb48140809&scope.project.id=5f8dc54c23f543d197444a43c6b82c01&role.id=eecd2f160dd844f5906f3d345780c501 => generated 259 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:12.712356 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad25454d-af45-4fb9-980b-4b47a2cd29d7 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:12.785671 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:12.834233 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad25454d-af45-4fb9-980b-4b47a2cd29d7 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:12.884943 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a58a97-fef0-4e12-ba16-1e57180d4134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:12.885604 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a58a97-fef0-4e12-ba16-1e57180d4134 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:12.885881 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a58a97-fef0-4e12-ba16-1e57180d4134 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:12.886149 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a58a97-fef0-4e12-ba16-1e57180d4134 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:12.903563 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a58a97-fef0-4e12-ba16-1e57180d4134 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:59:12.908684 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-54a58a97-fef0-4e12-ba16-1e57180d4134 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 11:59:12.908684 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:59:12.908684 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:59:12.908684 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:59:12.908684 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:59:12.908684 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:59:12.908684 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:59:12.908684 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:59:12.908684 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:59:12.908684 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:59:12.908684 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:59:12.908684 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:59:12.908684 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:59:12.908684 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:59:12.908684 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:59:12.908684 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:59:12.909810 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:59:12.909810 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:59:12.909810 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:59:12.909810 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:59:12.909810 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:59:12.909810 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:59:12.909810 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:59:12.909810 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:59:12.909810 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:59:12.909810 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:59:12.909810 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:59:12.909810 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 11:59:12.909810 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:59:12.909810 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:59:12.909810 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:59:12.909810 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:59:12.912454 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:59:12.912454 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:59:12.912454 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:59:12.912454 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:59:12.912454 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:59:12.912454 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:59:12.912454 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:59:12.912454 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:59:12.912454 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:59:12.912454 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:59:12.912454 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 11:59:12.912454 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 11:59:12.912454 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 118/225] 10.222.0.49 () {56 vars in 1114 bytes} [Fri Sep 9 11:59:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:59:12.920939 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bede0207-64a9-4582-b525-283eabc6935b None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:12.972482 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:13.039552 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a58a97-fef0-4e12-ba16-1e57180d4134 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:13.123011 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bede0207-64a9-4582-b525-283eabc6935b None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:13.222347 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a58a97-fef0-4e12-ba16-1e57180d4134 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:13.248367 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:13.248746 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac92267-58ee-4955-ac57-254cb6f6e864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:13.249975 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac92267-58ee-4955-ac57-254cb6f6e864 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:13.250509 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac92267-58ee-4955-ac57-254cb6f6e864 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:13.251306 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac92267-58ee-4955-ac57-254cb6f6e864 None admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:13.252562 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac92267-58ee-4955-ac57-254cb6f6e864 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:59:13.253048 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac92267-58ee-4955-ac57-254cb6f6e864 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:13.261825 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac92267-58ee-4955-ac57-254cb6f6e864 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:13.270831 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba790de5-c928-4997-be0f-b53009dc5147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:13.270831 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba790de5-c928-4997-be0f-b53009dc5147 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:13.270831 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba790de5-c928-4997-be0f-b53009dc5147 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:13.270831 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba790de5-c928-4997-be0f-b53009dc5147 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:13.302992 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 108/226] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 9 11:59:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 375 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:13.305762 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac92267-58ee-4955-ac57-254cb6f6e864 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:13.311145 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba790de5-c928-4997-be0f-b53009dc5147 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:59:13.329012 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-ba790de5-c928-4997-be0f-b53009dc5147 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:59:13.329012 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:59:13.329012 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:59:13.329012 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:59:13.329012 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:59:13.329012 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:59:13.329012 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:59:13.329012 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:59:13.329012 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:59:13.329012 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:59:13.329012 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:59:13.329012 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:59:13.329012 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:59:13.329012 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:59:13.329012 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:59:13.329012 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:59:13.329978 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:59:13.329978 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:59:13.329978 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:59:13.329978 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:59:13.329978 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:59:13.329978 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:59:13.329978 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:59:13.329978 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:59:13.329978 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:59:13.329978 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:59:13.329978 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:59:13.329978 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 11:59:13.329978 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:59:13.329978 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:59:13.329978 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:59:13.329978 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:59:13.330759 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:59:13.330759 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:59:13.330759 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:59:13.330759 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:59:13.330759 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:59:13.330759 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:59:13.330759 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:59:13.330759 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:59:13.330759 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:59:13.330759 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:59:13.330759 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:59:13.330759 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 11:59:13.330759 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 119/227] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Sep 9 11:59:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:59:13.341795 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba790de5-c928-4997-be0f-b53009dc5147 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:13.496358 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba790de5-c928-4997-be0f-b53009dc5147 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:13.543627 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:13.543820 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6544e32-963d-44a0-b270-c3a30900754d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:13.544176 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6544e32-963d-44a0-b270-c3a30900754d None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:13.544462 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6544e32-963d-44a0-b270-c3a30900754d None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:13.544731 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6544e32-963d-44a0-b270-c3a30900754d None admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:13.545452 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6544e32-963d-44a0-b270-c3a30900754d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:59:13.545756 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6544e32-963d-44a0-b270-c3a30900754d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:13.547209 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6544e32-963d-44a0-b270-c3a30900754d None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:13.557816 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 120/228] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 9 11:59:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:13.563278 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6544e32-963d-44a0-b270-c3a30900754d None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:13.587725 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac92267-58ee-4955-ac57-254cb6f6e864 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:13.652816 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:13.680213 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8173d5b5-c712-47d2-8de4-f6d157c8bc8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:13.680868 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8173d5b5-c712-47d2-8de4-f6d157c8bc8b None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:13.681150 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8173d5b5-c712-47d2-8de4-f6d157c8bc8b None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:13.681425 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8173d5b5-c712-47d2-8de4-f6d157c8bc8b None admin] PATH_INFO: `/v3/users/d4c01e0644c64ba1931a3d9db6b16207` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:13.686523 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8173d5b5-c712-47d2-8de4-f6d157c8bc8b None admin] RBAC: Authorizing `identity:get_user(user_id=d4c01e0644c64ba1931a3d9db6b16207)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:13.688027 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8173d5b5-c712-47d2-8de4-f6d157c8bc8b None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:13.690231 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 109/229] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Sep 9 11:59:13 2022] GET /identity/v3/users/d4c01e0644c64ba1931a3d9db6b16207 => generated 283 bytes in 386 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:13.695625 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8173d5b5-c712-47d2-8de4-f6d157c8bc8b None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:13.717089 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6544e32-963d-44a0-b270-c3a30900754d None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:13.758179 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014c1ecc-cf46-40f5-869c-5fbf9448ef7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:13.760494 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014c1ecc-cf46-40f5-869c-5fbf9448ef7d None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:13.761230 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014c1ecc-cf46-40f5-869c-5fbf9448ef7d None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:13.762757 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014c1ecc-cf46-40f5-869c-5fbf9448ef7d None admin] PATH_INFO: `/v3/users/d4c01e0644c64ba1931a3d9db6b16207` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:13.764362 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014c1ecc-cf46-40f5-869c-5fbf9448ef7d None admin] RBAC: Authorizing `identity:get_user(user_id=d4c01e0644c64ba1931a3d9db6b16207)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:13.766734 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014c1ecc-cf46-40f5-869c-5fbf9448ef7d None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:13.768742 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 121/230] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Sep 9 11:59:13 2022] GET /identity/v3/users/d4c01e0644c64ba1931a3d9db6b16207 => generated 283 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:13.780530 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014c1ecc-cf46-40f5-869c-5fbf9448ef7d None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:13.926700 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8173d5b5-c712-47d2-8de4-f6d157c8bc8b None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:13.951192 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014c1ecc-cf46-40f5-869c-5fbf9448ef7d None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:13.966318 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:13.991541 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed51ae08-e47a-48f1-a5f8-47362da0ad23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:13.992937 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed51ae08-e47a-48f1-a5f8-47362da0ad23 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:13.993223 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed51ae08-e47a-48f1-a5f8-47362da0ad23 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:13.993495 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed51ae08-e47a-48f1-a5f8-47362da0ad23 None admin] PATH_INFO: `/v3/projects/5f8dc54c23f543d197444a43c6b82c01` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:13.996024 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed51ae08-e47a-48f1-a5f8-47362da0ad23 None admin] RBAC: Authorizing `identity:get_project(project_id=5f8dc54c23f543d197444a43c6b82c01)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:13.997481 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed51ae08-e47a-48f1-a5f8-47362da0ad23 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:13.999322 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 110/231] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Sep 9 11:59:13 2022] GET /identity/v3/projects/5f8dc54c23f543d197444a43c6b82c01 => generated 302 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:14.009375 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6826b6dd-0808-411a-b803-bc7a94371575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:14.010059 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6826b6dd-0808-411a-b803-bc7a94371575 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:14.010338 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6826b6dd-0808-411a-b803-bc7a94371575 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:14.010661 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6826b6dd-0808-411a-b803-bc7a94371575 None admin] PATH_INFO: `/v3/projects/5f8dc54c23f543d197444a43c6b82c01` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:14.011819 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6826b6dd-0808-411a-b803-bc7a94371575 None admin] RBAC: Authorizing `identity:get_project(project_id=5f8dc54c23f543d197444a43c6b82c01)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:14.013264 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6826b6dd-0808-411a-b803-bc7a94371575 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:14.017344 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed51ae08-e47a-48f1-a5f8-47362da0ad23 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:14.027121 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 122/232] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Sep 9 11:59:13 2022] GET /identity/v3/projects/5f8dc54c23f543d197444a43c6b82c01 => generated 302 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:14.028032 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6826b6dd-0808-411a-b803-bc7a94371575 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:14.042642 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:14.163071 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed51ae08-e47a-48f1-a5f8-47362da0ad23 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:14.199819 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fe26c4-a2c1-4fa9-94f4-c2184c48a82c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:14.200785 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fe26c4-a2c1-4fa9-94f4-c2184c48a82c None admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:14.201498 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fe26c4-a2c1-4fa9-94f4-c2184c48a82c None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:14.201919 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fe26c4-a2c1-4fa9-94f4-c2184c48a82c None admin] PATH_INFO: `/v3/projects/5f8dc54c23f543d197444a43c6b82c01/users/d4c01e0644c64ba1931a3d9db6b16207/roles/a7c71d2d9914408bb9953f85329d1e80` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:14.204666 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fe26c4-a2c1-4fa9-94f4-c2184c48a82c None admin] RBAC: Authorizing `identity:create_grant(project_id=5f8dc54c23f543d197444a43c6b82c01, user_id=d4c01e0644c64ba1931a3d9db6b16207, role_id=a7c71d2d9914408bb9953f85329d1e80)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:14.206402 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fe26c4-a2c1-4fa9-94f4-c2184c48a82c None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:14.234988 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 111/233] 10.222.0.49 () {58 vars in 1439 bytes} [Fri Sep 9 11:59:14 2022] PUT /identity/v3/projects/5f8dc54c23f543d197444a43c6b82c01/users/d4c01e0644c64ba1931a3d9db6b16207/roles/a7c71d2d9914408bb9953f85329d1e80 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:59:14.276092 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6826b6dd-0808-411a-b803-bc7a94371575 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:14.323643 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beee572f-d8c5-47db-88cf-25b9cbc00480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:14.324561 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beee572f-d8c5-47db-88cf-25b9cbc00480 None admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:14.324986 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beee572f-d8c5-47db-88cf-25b9cbc00480 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:14.325403 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beee572f-d8c5-47db-88cf-25b9cbc00480 None admin] PATH_INFO: `/v3/projects/5f8dc54c23f543d197444a43c6b82c01/users/d4c01e0644c64ba1931a3d9db6b16207/roles/3f2ca33902a04b6dbc2ef7d06d7289d3` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:14.327639 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beee572f-d8c5-47db-88cf-25b9cbc00480 None admin] RBAC: Authorizing `identity:create_grant(project_id=5f8dc54c23f543d197444a43c6b82c01, user_id=d4c01e0644c64ba1931a3d9db6b16207, role_id=3f2ca33902a04b6dbc2ef7d06d7289d3)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:14.329222 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beee572f-d8c5-47db-88cf-25b9cbc00480 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:14.348408 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 123/234] 10.222.0.49 () {58 vars in 1439 bytes} [Fri Sep 9 11:59:14 2022] PUT /identity/v3/projects/5f8dc54c23f543d197444a43c6b82c01/users/d4c01e0644c64ba1931a3d9db6b16207/roles/3f2ca33902a04b6dbc2ef7d06d7289d3 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:59:14.688448 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a18cbd-54c4-4442-b99a-52419f7cbbc2 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:14.689665 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a18cbd-54c4-4442-b99a-52419f7cbbc2 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:14.690148 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a18cbd-54c4-4442-b99a-52419f7cbbc2 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:14.691728 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 112/235] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 11:59:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:59:14.697461 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dc7e35-79a2-406b-934a-1f0eeff0f295 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:14.697976 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dc7e35-79a2-406b-934a-1f0eeff0f295 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:14.698461 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dc7e35-79a2-406b-934a-1f0eeff0f295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:14.720135 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-38dc7e35-79a2-406b-934a-1f0eeff0f295 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:59:14.721642 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-38dc7e35-79a2-406b-934a-1f0eeff0f295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:14.761304 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:14.803219 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 124/236] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 11:59:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:59:14.941114 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbea773-e0a5-4bd5-8514-83f4aaf7b009 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:14.941731 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbea773-e0a5-4bd5-8514-83f4aaf7b009 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:14.942201 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbea773-e0a5-4bd5-8514-83f4aaf7b009 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:14.944366 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 113/237] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 11:59:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:59:14.949772 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8e8803-8536-4a2b-8569-5ce6e05e82b2 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:14.950306 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8e8803-8536-4a2b-8569-5ce6e05e82b2 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:14.950796 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8e8803-8536-4a2b-8569-5ce6e05e82b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:14.973306 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-2f8e8803-8536-4a2b-8569-5ce6e05e82b2 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:59:14.975112 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-2f8e8803-8536-4a2b-8569-5ce6e05e82b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:15.051755 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 125/238] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 11:59:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:59:15.155186 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbea773-e0a5-4bd5-8514-83f4aaf7b009 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:15.265518 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:15.301631 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbea773-e0a5-4bd5-8514-83f4aaf7b009 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:15.346766 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44113860-3f7e-4396-8a82-ccfb15669ab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:15.347439 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44113860-3f7e-4396-8a82-ccfb15669ab0 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:15.347717 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44113860-3f7e-4396-8a82-ccfb15669ab0 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:15.347985 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44113860-3f7e-4396-8a82-ccfb15669ab0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:15.362681 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44113860-3f7e-4396-8a82-ccfb15669ab0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:59:15.368642 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-44113860-3f7e-4396-8a82-ccfb15669ab0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 11:59:15.368642 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:59:15.368642 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:59:15.368642 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:59:15.368642 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:59:15.368642 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:59:15.368642 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:59:15.368642 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:59:15.368642 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:59:15.368642 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:59:15.368642 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:59:15.368642 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:59:15.368642 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:59:15.368642 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:59:15.368642 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:59:15.368642 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:59:15.369519 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:59:15.369519 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:59:15.369519 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:59:15.369519 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:59:15.369519 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:59:15.369519 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:59:15.369519 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:59:15.369519 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:59:15.369519 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:59:15.369519 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:59:15.369519 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:59:15.369519 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 11:59:15.369519 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:59:15.369519 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:59:15.369519 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:59:15.369519 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:59:15.370301 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:59:15.370301 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:59:15.370301 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:59:15.370301 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:59:15.370301 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:59:15.370301 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:59:15.370301 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:59:15.370301 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:59:15.370301 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:59:15.370301 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:59:15.370301 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 11:59:15.370301 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 11:59:15.370301 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 114/239] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 9 11:59:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:59:15.376799 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8e8803-8536-4a2b-8569-5ce6e05e82b2 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:15.484192 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:15.494271 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8e8803-8536-4a2b-8569-5ce6e05e82b2 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:15.510645 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44113860-3f7e-4396-8a82-ccfb15669ab0 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:15.562712 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41edd5fa-286c-4bc2-adc6-9d564880f148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:15.563496 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41edd5fa-286c-4bc2-adc6-9d564880f148 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:15.563770 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41edd5fa-286c-4bc2-adc6-9d564880f148 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:15.564284 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41edd5fa-286c-4bc2-adc6-9d564880f148 None admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:15.564958 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41edd5fa-286c-4bc2-adc6-9d564880f148 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:59:15.565252 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41edd5fa-286c-4bc2-adc6-9d564880f148 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:15.566624 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41edd5fa-286c-4bc2-adc6-9d564880f148 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:15.589358 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 126/240] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Sep 9 11:59:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:15.594553 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271e2f4c-214d-4469-8081-60eb12e07680 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:15.594843 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271e2f4c-214d-4469-8081-60eb12e07680 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:15.595150 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271e2f4c-214d-4469-8081-60eb12e07680 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:15.596546 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 127/241] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 11:59:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 11:59:15.597119 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271e2f4c-214d-4469-8081-60eb12e07680 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:15.663771 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44113860-3f7e-4396-8a82-ccfb15669ab0 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:15.702197 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d0dd95-ac3e-4d71-9320-7c2b20edf0ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:15.703674 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d0dd95-ac3e-4d71-9320-7c2b20edf0ca None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:15.703964 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d0dd95-ac3e-4d71-9320-7c2b20edf0ca None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:15.704339 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d0dd95-ac3e-4d71-9320-7c2b20edf0ca None admin] PATH_INFO: `/v3/domains/default` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:15.707821 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d0dd95-ac3e-4d71-9320-7c2b20edf0ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:15.710203 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d0dd95-ac3e-4d71-9320-7c2b20edf0ca None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:15.712524 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 115/242] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 11:59:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:15.714236 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07e31ca-021d-4178-af4b-c1f1a5e43a2f None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:15.714701 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07e31ca-021d-4178-af4b-c1f1a5e43a2f None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:15.715151 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07e31ca-021d-4178-af4b-c1f1a5e43a2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:15.721055 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271e2f4c-214d-4469-8081-60eb12e07680 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:15.724403 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:15.740528 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-a07e31ca-021d-4178-af4b-c1f1a5e43a2f None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:59:15.742397 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-a07e31ca-021d-4178-af4b-c1f1a5e43a2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:15.763291 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e997173b-f121-4d25-8754-b601ff76b174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:15.764059 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e997173b-f121-4d25-8754-b601ff76b174 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:15.764398 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e997173b-f121-4d25-8754-b601ff76b174 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:15.764800 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e997173b-f121-4d25-8754-b601ff76b174 None admin] PATH_INFO: `/v3/users/eab6a106413447d3bc8181a05ed74713` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:15.779153 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e997173b-f121-4d25-8754-b601ff76b174 None admin] RBAC: Authorizing `identity:get_user(user_id=eab6a106413447d3bc8181a05ed74713)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:15.780681 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e997173b-f121-4d25-8754-b601ff76b174 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:15.782600 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 128/243] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Sep 9 11:59:15 2022] GET /identity/v3/users/eab6a106413447d3bc8181a05ed74713 => generated 297 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:15.788603 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e997173b-f121-4d25-8754-b601ff76b174 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:15.811674 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:15.838359 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 116/244] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 11:59:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:59:15.839784 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07e31ca-021d-4178-af4b-c1f1a5e43a2f None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:15.932488 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e997173b-f121-4d25-8754-b601ff76b174 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:15.958178 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07e31ca-021d-4178-af4b-c1f1a5e43a2f None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:15.996438 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2632c2-e753-4fdc-8acb-5c4c37b9dec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:15.997354 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2632c2-e753-4fdc-8acb-5c4c37b9dec8 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:15.997766 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2632c2-e753-4fdc-8acb-5c4c37b9dec8 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:15.998182 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2632c2-e753-4fdc-8acb-5c4c37b9dec8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:16.001005 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:16.016162 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fa4857-15d4-4fb5-b95b-d30e8b6576df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:16.016777 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fa4857-15d4-4fb5-b95b-d30e8b6576df None admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:16.017060 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fa4857-15d4-4fb5-b95b-d30e8b6576df None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:16.017543 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fa4857-15d4-4fb5-b95b-d30e8b6576df None admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:16.017967 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fa4857-15d4-4fb5-b95b-d30e8b6576df None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:16.019445 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fa4857-15d4-4fb5-b95b-d30e8b6576df None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:16.076592 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2632c2-e753-4fdc-8acb-5c4c37b9dec8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:16.078517 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2632c2-e753-4fdc-8acb-5c4c37b9dec8 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:16.085959 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 129/245] 10.222.0.49 () {60 vars in 1132 bytes} [Fri Sep 9 11:59:15 2022] POST /identity/v3/users => generated 293 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:59:16.206466 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fa4857-15d4-4fb5-b95b-d30e8b6576df None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:16.221205 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 117/246] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 9 11:59:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 11:59:16.232102 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2632c2-e753-4fdc-8acb-5c4c37b9dec8 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:16.300415 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:16.308517 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:16.336414 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fa4857-15d4-4fb5-b95b-d30e8b6576df None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:16.374528 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f8045b-3886-498a-889f-e3fa1fc8b946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:16.375608 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f8045b-3886-498a-889f-e3fa1fc8b946 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:16.376100 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f8045b-3886-498a-889f-e3fa1fc8b946 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:16.376563 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f8045b-3886-498a-889f-e3fa1fc8b946 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:16.388618 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f8045b-3886-498a-889f-e3fa1fc8b946 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:59:16.395057 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-84f8045b-3886-498a-889f-e3fa1fc8b946 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 11:59:16.395057 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:59:16.395057 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:59:16.395057 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:59:16.395057 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:59:16.395057 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:59:16.395057 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:59:16.395057 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:59:16.395057 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:59:16.395057 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:59:16.395057 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:59:16.395057 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:59:16.395057 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:59:16.395057 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:59:16.395057 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:59:16.395057 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:59:16.396528 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:59:16.396528 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:59:16.396528 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:59:16.396528 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:59:16.396528 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:59:16.396528 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:59:16.396528 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:59:16.396528 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:59:16.396528 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:59:16.396528 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:59:16.396528 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:59:16.396528 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 11:59:16.396528 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:59:16.396528 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:59:16.396528 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:59:16.396528 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:59:16.397797 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:59:16.397797 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:59:16.397797 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:59:16.397797 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:59:16.397797 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:59:16.397797 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:59:16.397797 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:59:16.397797 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:59:16.397797 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:59:16.397797 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:59:16.397797 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 11:59:16.397797 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 11:59:16.402241 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 130/247] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 9 11:59:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:59:16.408016 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f8045b-3886-498a-889f-e3fa1fc8b946 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:16.434579 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2632c2-e753-4fdc-8acb-5c4c37b9dec8 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:16.481266 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff397c54-f824-4e26-9cef-330a3c40906f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:16.481888 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff397c54-f824-4e26-9cef-330a3c40906f None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:16.482172 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff397c54-f824-4e26-9cef-330a3c40906f None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:16.482445 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff397c54-f824-4e26-9cef-330a3c40906f None admin] PATH_INFO: `/v3/projects/5f8dc54c23f543d197444a43c6b82c01` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:16.484550 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff397c54-f824-4e26-9cef-330a3c40906f None admin] RBAC: Authorizing `identity:get_project(project_id=5f8dc54c23f543d197444a43c6b82c01)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:16.486124 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff397c54-f824-4e26-9cef-330a3c40906f None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:16.487880 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 118/248] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Sep 9 11:59:16 2022] GET /identity/v3/projects/5f8dc54c23f543d197444a43c6b82c01 => generated 302 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:16.494273 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff397c54-f824-4e26-9cef-330a3c40906f None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:16.527122 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f8045b-3886-498a-889f-e3fa1fc8b946 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:16.563848 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087e2bba-119b-4a3d-9b42-89ebf5b3655c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:16.564845 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087e2bba-119b-4a3d-9b42-89ebf5b3655c None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:16.565185 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087e2bba-119b-4a3d-9b42-89ebf5b3655c None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:16.565511 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087e2bba-119b-4a3d-9b42-89ebf5b3655c None admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:16.566435 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087e2bba-119b-4a3d-9b42-89ebf5b3655c None admin] RBAC: Adding query filter params (name=member) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:59:16.566552 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087e2bba-119b-4a3d-9b42-89ebf5b3655c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:16.568969 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087e2bba-119b-4a3d-9b42-89ebf5b3655c None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:16.571732 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:16.582570 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 131/249] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Sep 9 11:59:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:16.591944 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087e2bba-119b-4a3d-9b42-89ebf5b3655c None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:16.616250 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff397c54-f824-4e26-9cef-330a3c40906f None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:16.618287 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:16.698636 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1d9231-f70a-463e-a4b6-b2c12b96f8e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:16.699840 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1d9231-f70a-463e-a4b6-b2c12b96f8e6 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:16.700122 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1d9231-f70a-463e-a4b6-b2c12b96f8e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:16.700662 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1d9231-f70a-463e-a4b6-b2c12b96f8e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:16.701399 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1d9231-f70a-463e-a4b6-b2c12b96f8e6 None admin] RBAC: Adding query filter params (role.id=b1e7a1e6ddc54da7ba995fb55fd93016, scope.project.id=5f8dc54c23f543d197444a43c6b82c01, user.id=eab6a106413447d3bc8181a05ed74713) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:59:16.701861 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1d9231-f70a-463e-a4b6-b2c12b96f8e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:16.703945 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1d9231-f70a-463e-a4b6-b2c12b96f8e6 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:16.713245 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 119/250] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Sep 9 11:59:16 2022] GET /identity/v3/role_assignments?user.id=eab6a106413447d3bc8181a05ed74713&scope.project.id=5f8dc54c23f543d197444a43c6b82c01&role.id=b1e7a1e6ddc54da7ba995fb55fd93016 => generated 259 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:59:16.713754 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087e2bba-119b-4a3d-9b42-89ebf5b3655c None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:16.770254 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8c99d4-57d8-41dd-a419-7a72507e7e8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:16.771514 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8c99d4-57d8-41dd-a419-7a72507e7e8c None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:16.771853 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8c99d4-57d8-41dd-a419-7a72507e7e8c None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:16.772253 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8c99d4-57d8-41dd-a419-7a72507e7e8c None admin] PATH_INFO: `/v3/users/976211872e2f4f188fa5e2eb48140809` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:16.773526 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8c99d4-57d8-41dd-a419-7a72507e7e8c None admin] RBAC: Authorizing `identity:get_user(user_id=976211872e2f4f188fa5e2eb48140809)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:16.775047 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8c99d4-57d8-41dd-a419-7a72507e7e8c None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:16.779070 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 132/251] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Sep 9 11:59:16 2022] GET /identity/v3/users/976211872e2f4f188fa5e2eb48140809 => generated 297 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:16.782482 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1d9231-f70a-463e-a4b6-b2c12b96f8e6 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:16.907124 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1d9231-f70a-463e-a4b6-b2c12b96f8e6 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:16.950787 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83f01e0-6979-44b5-89ef-069a31fd3edf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:16.951494 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83f01e0-6979-44b5-89ef-069a31fd3edf None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:16.951770 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83f01e0-6979-44b5-89ef-069a31fd3edf None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:16.952041 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83f01e0-6979-44b5-89ef-069a31fd3edf None admin] PATH_INFO: `/v3/projects/5f8dc54c23f543d197444a43c6b82c01` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:16.953299 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83f01e0-6979-44b5-89ef-069a31fd3edf None admin] RBAC: Authorizing `identity:get_project(project_id=5f8dc54c23f543d197444a43c6b82c01)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:16.954816 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83f01e0-6979-44b5-89ef-069a31fd3edf None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:16.957182 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 120/252] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Sep 9 11:59:16 2022] GET /identity/v3/projects/5f8dc54c23f543d197444a43c6b82c01 => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:16.963574 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8c99d4-57d8-41dd-a419-7a72507e7e8c None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:16.991590 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f78be9e-78e6-479b-9f84-489f5a845110 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:16.991887 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f78be9e-78e6-479b-9f84-489f5a845110 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:16.992134 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f78be9e-78e6-479b-9f84-489f5a845110 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:16.993755 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 121/253] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 11:59:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 11:59:17.000036 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63364569-6f6c-4f69-835e-9112f08953a9 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:17.000447 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63364569-6f6c-4f69-835e-9112f08953a9 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:17.000722 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63364569-6f6c-4f69-835e-9112f08953a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:17.026305 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-63364569-6f6c-4f69-835e-9112f08953a9 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:59:17.027787 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-63364569-6f6c-4f69-835e-9112f08953a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:17.047393 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:17.070129 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:17.112211 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8c99d4-57d8-41dd-a419-7a72507e7e8c None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:17.154417 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 122/254] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 11:59:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:59:17.155757 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42e7285-6d13-4ca8-9800-75fa40d95331 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:17.156037 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42e7285-6d13-4ca8-9800-75fa40d95331 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:17.156306 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42e7285-6d13-4ca8-9800-75fa40d95331 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:17.157715 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 123/255] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 11:59:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:59:17.164265 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fa04d1-2a3f-4f39-8345-71545944eb35 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:17.164539 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fa04d1-2a3f-4f39-8345-71545944eb35 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:17.164801 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fa04d1-2a3f-4f39-8345-71545944eb35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:17.171417 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3c54c4-f3b3-46ae-aec2-1e90abc44a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:17.172085 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3c54c4-f3b3-46ae-aec2-1e90abc44a5e None admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:17.172356 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3c54c4-f3b3-46ae-aec2-1e90abc44a5e None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:17.172623 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3c54c4-f3b3-46ae-aec2-1e90abc44a5e None admin] PATH_INFO: `/v3/projects/5f8dc54c23f543d197444a43c6b82c01/users/976211872e2f4f188fa5e2eb48140809/roles/eecd2f160dd844f5906f3d345780c501` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:17.175181 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3c54c4-f3b3-46ae-aec2-1e90abc44a5e None admin] RBAC: Authorizing `identity:create_grant(project_id=5f8dc54c23f543d197444a43c6b82c01, user_id=976211872e2f4f188fa5e2eb48140809, role_id=eecd2f160dd844f5906f3d345780c501)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:17.176628 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3c54c4-f3b3-46ae-aec2-1e90abc44a5e None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:17.201609 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 133/256] 10.222.0.49 () {58 vars in 1439 bytes} [Fri Sep 9 11:59:16 2022] PUT /identity/v3/projects/5f8dc54c23f543d197444a43c6b82c01/users/976211872e2f4f188fa5e2eb48140809/roles/eecd2f160dd844f5906f3d345780c501 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 11:59:17.203260 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-90fa04d1-2a3f-4f39-8345-71545944eb35 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:59:17.204513 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-90fa04d1-2a3f-4f39-8345-71545944eb35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:17.299861 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 124/257] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 11:59:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 11:59:17.585125 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3c54c4-f3b3-46ae-aec2-1e90abc44a5e None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:17.687982 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fa04d1-2a3f-4f39-8345-71545944eb35 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:17.733644 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3c54c4-f3b3-46ae-aec2-1e90abc44a5e None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:17.786635 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:17.823627 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e19edb-fc3a-4020-8edf-91de47667e58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:17.824278 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e19edb-fc3a-4020-8edf-91de47667e58 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:17.824562 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e19edb-fc3a-4020-8edf-91de47667e58 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:17.824830 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e19edb-fc3a-4020-8edf-91de47667e58 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:17.855971 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e19edb-fc3a-4020-8edf-91de47667e58 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:59:17.861944 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-22e19edb-fc3a-4020-8edf-91de47667e58 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 11:59:17.861944 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:59:17.861944 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:59:17.861944 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:59:17.861944 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:59:17.861944 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:59:17.861944 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:59:17.861944 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:59:17.861944 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:59:17.861944 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:59:17.861944 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:59:17.861944 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:59:17.861944 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:59:17.861944 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:59:17.861944 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:59:17.861944 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:59:17.862810 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:59:17.862810 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:59:17.862810 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:59:17.862810 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:59:17.862810 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:59:17.862810 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:59:17.862810 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:59:17.862810 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:59:17.862810 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:59:17.862810 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:59:17.862810 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:59:17.862810 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 11:59:17.862810 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:59:17.862810 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:59:17.862810 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:59:17.862810 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:59:17.863645 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:59:17.863645 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:59:17.863645 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:59:17.863645 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:59:17.863645 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:59:17.863645 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:59:17.863645 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:59:17.863645 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:59:17.863645 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:59:17.863645 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:59:17.863645 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 11:59:17.863645 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 11:59:17.863645 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 134/258] 10.222.0.49 () {56 vars in 1114 bytes} [Fri Sep 9 11:59:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:59:17.872693 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e19edb-fc3a-4020-8edf-91de47667e58 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:17.881344 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:17.899901 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fa04d1-2a3f-4f39-8345-71545944eb35 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:17.963623 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ca50e3-3c1a-457b-9de4-16ded3ca4549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:17.964665 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ca50e3-3c1a-457b-9de4-16ded3ca4549 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:17.965150 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ca50e3-3c1a-457b-9de4-16ded3ca4549 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:17.965618 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ca50e3-3c1a-457b-9de4-16ded3ca4549 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:17.979990 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ca50e3-3c1a-457b-9de4-16ded3ca4549 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:59:17.990943 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-72ca50e3-3c1a-457b-9de4-16ded3ca4549 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:59:17.990943 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:59:17.990943 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:59:17.990943 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:59:17.990943 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:59:17.990943 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:59:17.990943 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:59:17.990943 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:59:17.990943 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:59:17.990943 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:59:17.990943 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:59:17.990943 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:59:17.990943 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:59:17.990943 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:59:17.990943 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:59:17.990943 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:59:17.991932 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:59:17.991932 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:59:17.991932 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:59:17.991932 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:59:17.991932 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:59:17.991932 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:59:17.991932 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:59:17.991932 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:59:17.991932 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:59:17.991932 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:59:17.991932 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:59:17.991932 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 11:59:17.991932 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:59:17.991932 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:59:17.991932 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:59:17.991932 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:59:17.992724 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:59:17.992724 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:59:17.992724 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:59:17.992724 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:59:17.992724 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:59:17.992724 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:59:17.992724 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:59:17.992724 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:59:17.992724 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:59:17.992724 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:59:17.992724 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:59:17.992724 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 11:59:17.995545 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 125/259] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Sep 9 11:59:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:59:18.003196 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ca50e3-3c1a-457b-9de4-16ded3ca4549 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:18.144738 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e19edb-fc3a-4020-8edf-91de47667e58 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:18.180026 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ca50e3-3c1a-457b-9de4-16ded3ca4549 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:18.198911 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abea9c34-bc81-4602-8353-6ad3aea3c2ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:18.199524 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abea9c34-bc81-4602-8353-6ad3aea3c2ca None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:18.199796 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abea9c34-bc81-4602-8353-6ad3aea3c2ca None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:18.200060 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abea9c34-bc81-4602-8353-6ad3aea3c2ca None admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:18.200717 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abea9c34-bc81-4602-8353-6ad3aea3c2ca None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:59:18.201011 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abea9c34-bc81-4602-8353-6ad3aea3c2ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:18.202372 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abea9c34-bc81-4602-8353-6ad3aea3c2ca None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:18.219745 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 135/260] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 9 11:59:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:59:18.226575 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abea9c34-bc81-4602-8353-6ad3aea3c2ca None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:18.252436 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed86cf6-2f85-4083-a613-6b9c18bc2f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:18.253056 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed86cf6-2f85-4083-a613-6b9c18bc2f60 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:18.253333 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed86cf6-2f85-4083-a613-6b9c18bc2f60 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:18.253602 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed86cf6-2f85-4083-a613-6b9c18bc2f60 None admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:18.254270 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed86cf6-2f85-4083-a613-6b9c18bc2f60 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:59:18.254576 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed86cf6-2f85-4083-a613-6b9c18bc2f60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:18.256585 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed86cf6-2f85-4083-a613-6b9c18bc2f60 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:18.270002 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 126/261] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 9 11:59:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:59:18.281866 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed86cf6-2f85-4083-a613-6b9c18bc2f60 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:18.306329 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:18.314008 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:18.390644 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abea9c34-bc81-4602-8353-6ad3aea3c2ca None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:18.459696 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9b3b15-1638-405a-8f04-0d8d891c0d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:18.460562 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9b3b15-1638-405a-8f04-0d8d891c0d22 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:18.460981 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9b3b15-1638-405a-8f04-0d8d891c0d22 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:18.461386 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9b3b15-1638-405a-8f04-0d8d891c0d22 None admin] PATH_INFO: `/v3/users/d4c01e0644c64ba1931a3d9db6b16207` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:18.462694 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9b3b15-1638-405a-8f04-0d8d891c0d22 None admin] RBAC: Authorizing `identity:get_user(user_id=d4c01e0644c64ba1931a3d9db6b16207)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:18.464328 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9b3b15-1638-405a-8f04-0d8d891c0d22 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:18.466446 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 136/262] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Sep 9 11:59:18 2022] GET /identity/v3/users/d4c01e0644c64ba1931a3d9db6b16207 => generated 283 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:59:18.476079 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9b3b15-1638-405a-8f04-0d8d891c0d22 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:18.489466 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed86cf6-2f85-4083-a613-6b9c18bc2f60 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:18.556221 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369f0a8a-5ee8-4e09-937a-3f9bd1c72f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:18.557216 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369f0a8a-5ee8-4e09-937a-3f9bd1c72f8f None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:18.557636 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369f0a8a-5ee8-4e09-937a-3f9bd1c72f8f None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:18.558048 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369f0a8a-5ee8-4e09-937a-3f9bd1c72f8f None admin] PATH_INFO: `/v3/users/d4c01e0644c64ba1931a3d9db6b16207` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:18.563732 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369f0a8a-5ee8-4e09-937a-3f9bd1c72f8f None admin] RBAC: Authorizing `identity:get_user(user_id=d4c01e0644c64ba1931a3d9db6b16207)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:18.565533 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369f0a8a-5ee8-4e09-937a-3f9bd1c72f8f None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:18.571067 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 127/263] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Sep 9 11:59:18 2022] GET /identity/v3/users/d4c01e0644c64ba1931a3d9db6b16207 => generated 283 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:59:18.592502 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369f0a8a-5ee8-4e09-937a-3f9bd1c72f8f None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:18.650164 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9b3b15-1638-405a-8f04-0d8d891c0d22 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:18.687240 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:18.715712 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025c23ad-c7da-41ee-b4bf-d9311ed4d023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:18.716654 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025c23ad-c7da-41ee-b4bf-d9311ed4d023 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:18.717128 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025c23ad-c7da-41ee-b4bf-d9311ed4d023 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:18.717773 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025c23ad-c7da-41ee-b4bf-d9311ed4d023 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:18.769274 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:18.794713 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369f0a8a-5ee8-4e09-937a-3f9bd1c72f8f None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:18.845630 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025c23ad-c7da-41ee-b4bf-d9311ed4d023 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:18.847469 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025c23ad-c7da-41ee-b4bf-d9311ed4d023 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:18.855583 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba89b2f-9b37-460e-a040-c29a516aee22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:18.856431 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba89b2f-9b37-460e-a040-c29a516aee22 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:18.856885 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba89b2f-9b37-460e-a040-c29a516aee22 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:18.857301 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba89b2f-9b37-460e-a040-c29a516aee22 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:18.973269 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba89b2f-9b37-460e-a040-c29a516aee22 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:18.976679 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba89b2f-9b37-460e-a040-c29a516aee22 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:19.049029 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 137/264] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 9 11:59:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 573 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 09 11:59:19.054382 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025c23ad-c7da-41ee-b4bf-d9311ed4d023 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:19.093850 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:19.112718 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:19.133323 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 128/265] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 9 11:59:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 541 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 09 11:59:19.140909 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba89b2f-9b37-460e-a040-c29a516aee22 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:19.272842 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025c23ad-c7da-41ee-b4bf-d9311ed4d023 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:19.331631 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd1a76f-ff06-4f14-bf85-e12d9a21b229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:19.332277 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd1a76f-ff06-4f14-bf85-e12d9a21b229 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:19.332554 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd1a76f-ff06-4f14-bf85-e12d9a21b229 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:19.332827 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd1a76f-ff06-4f14-bf85-e12d9a21b229 None admin] PATH_INFO: `/v3/projects/5f8dc54c23f543d197444a43c6b82c01` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:19.334656 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd1a76f-ff06-4f14-bf85-e12d9a21b229 None admin] RBAC: Authorizing `identity:get_project(project_id=5f8dc54c23f543d197444a43c6b82c01)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:19.336241 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd1a76f-ff06-4f14-bf85-e12d9a21b229 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:19.338201 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 138/266] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Sep 9 11:59:19 2022] GET /identity/v3/projects/5f8dc54c23f543d197444a43c6b82c01 => generated 302 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:59:19.343695 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd1a76f-ff06-4f14-bf85-e12d9a21b229 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:19.398001 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba89b2f-9b37-460e-a040-c29a516aee22 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:19.484367 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:19.492903 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd1a76f-ff06-4f14-bf85-e12d9a21b229 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:19.515648 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe69681-94b4-4627-ad25-9d80a26a3f9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:19.516300 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe69681-94b4-4627-ad25-9d80a26a3f9d None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:19.516580 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe69681-94b4-4627-ad25-9d80a26a3f9d None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:19.516851 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe69681-94b4-4627-ad25-9d80a26a3f9d None admin] PATH_INFO: `/v3/projects/5f8dc54c23f543d197444a43c6b82c01` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:19.523693 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe69681-94b4-4627-ad25-9d80a26a3f9d None admin] RBAC: Authorizing `identity:get_project(project_id=5f8dc54c23f543d197444a43c6b82c01)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:19.525098 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe69681-94b4-4627-ad25-9d80a26a3f9d None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:19.527269 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 129/267] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Sep 9 11:59:19 2022] GET /identity/v3/projects/5f8dc54c23f543d197444a43c6b82c01 => generated 302 bytes in 387 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:59:19.534549 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe5c7e6-4c48-4c9f-96b0-4221f5c0a860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:19.535065 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe69681-94b4-4627-ad25-9d80a26a3f9d None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:19.535649 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe5c7e6-4c48-4c9f-96b0-4221f5c0a860 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:19.536065 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe5c7e6-4c48-4c9f-96b0-4221f5c0a860 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:19.536606 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe5c7e6-4c48-4c9f-96b0-4221f5c0a860 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:19.538065 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe5c7e6-4c48-4c9f-96b0-4221f5c0a860 None admin] RBAC: Adding query filter params (role.id=a7c71d2d9914408bb9953f85329d1e80, scope.project.id=5f8dc54c23f543d197444a43c6b82c01, user.id=d4c01e0644c64ba1931a3d9db6b16207) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:59:19.538517 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe5c7e6-4c48-4c9f-96b0-4221f5c0a860 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:19.540176 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe5c7e6-4c48-4c9f-96b0-4221f5c0a860 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:19.547156 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 139/268] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Sep 9 11:59:19 2022] GET /identity/v3/role_assignments?user.id=d4c01e0644c64ba1931a3d9db6b16207&scope.project.id=5f8dc54c23f543d197444a43c6b82c01&role.id=a7c71d2d9914408bb9953f85329d1e80 => generated 608 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:59:19.609737 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:19.686717 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe69681-94b4-4627-ad25-9d80a26a3f9d None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:19.715432 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f411f6c-e470-4c99-bd83-71c7be8de1f9 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:19.716410 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f411f6c-e470-4c99-bd83-71c7be8de1f9 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:19.716863 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f411f6c-e470-4c99-bd83-71c7be8de1f9 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:19.718698 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 140/269] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 11:59:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:59:19.725454 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e126e4-0b30-4553-992a-947db4f50c6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:19.726083 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e126e4-0b30-4553-992a-947db4f50c6f None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:19.726362 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e126e4-0b30-4553-992a-947db4f50c6f None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:19.726705 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e126e4-0b30-4553-992a-947db4f50c6f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:19.726979 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd476fa-5e3c-4bc8-a77a-8393c762ed86 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:19.727451 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd476fa-5e3c-4bc8-a77a-8393c762ed86 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:19.727550 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e126e4-0b30-4553-992a-947db4f50c6f None admin] RBAC: Adding query filter params (role.id=3f2ca33902a04b6dbc2ef7d06d7289d3, scope.project.id=5f8dc54c23f543d197444a43c6b82c01, user.id=d4c01e0644c64ba1931a3d9db6b16207) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:59:19.727840 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e126e4-0b30-4553-992a-947db4f50c6f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:19.728205 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd476fa-5e3c-4bc8-a77a-8393c762ed86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:19.729346 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e126e4-0b30-4553-992a-947db4f50c6f None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:19.736586 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 130/270] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Sep 9 11:59:19 2022] GET /identity/v3/role_assignments?user.id=d4c01e0644c64ba1931a3d9db6b16207&scope.project.id=5f8dc54c23f543d197444a43c6b82c01&role.id=3f2ca33902a04b6dbc2ef7d06d7289d3 => generated 608 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:19.760128 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-9cd476fa-5e3c-4bc8-a77a-8393c762ed86 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:59:19.761626 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-9cd476fa-5e3c-4bc8-a77a-8393c762ed86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:19.856947 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 141/271] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 11:59:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 11:59:19.936311 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d702a4b7-d4ef-4136-a47e-ed2cc770408f None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:19.936962 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d702a4b7-d4ef-4136-a47e-ed2cc770408f None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:19.937438 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d702a4b7-d4ef-4136-a47e-ed2cc770408f None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:19.939115 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 131/272] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 11:59:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:59:19.945467 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4494f493-6dc6-4001-b396-c027f2c85a47 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:19.945740 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4494f493-6dc6-4001-b396-c027f2c85a47 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:19.946017 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4494f493-6dc6-4001-b396-c027f2c85a47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:19.972910 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-4494f493-6dc6-4001-b396-c027f2c85a47 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:59:19.974790 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-4494f493-6dc6-4001-b396-c027f2c85a47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:20.069659 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 142/273] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 11:59:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:59:20.104888 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861bb304-0025-48ab-94ad-a77f33376b56 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:20.105858 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861bb304-0025-48ab-94ad-a77f33376b56 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:20.106335 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861bb304-0025-48ab-94ad-a77f33376b56 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:20.108246 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 132/274] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 11:59:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:59:20.113393 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879e93c4-89f6-4ee1-8303-3498f8d2dd99 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:20.113673 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879e93c4-89f6-4ee1-8303-3498f8d2dd99 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:20.113945 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879e93c4-89f6-4ee1-8303-3498f8d2dd99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:20.135697 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-879e93c4-89f6-4ee1-8303-3498f8d2dd99 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:59:20.136893 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-879e93c4-89f6-4ee1-8303-3498f8d2dd99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:20.138970 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:20.217688 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 143/275] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 11:59:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:59:20.222724 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861bb304-0025-48ab-94ad-a77f33376b56 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:20.344078 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861bb304-0025-48ab-94ad-a77f33376b56 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:20.363555 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:20.385187 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71edbacf-7d36-45fe-8f37-9daa5c6b79ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:20.386185 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71edbacf-7d36-45fe-8f37-9daa5c6b79ab None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:20.386652 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71edbacf-7d36-45fe-8f37-9daa5c6b79ab None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:20.387131 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71edbacf-7d36-45fe-8f37-9daa5c6b79ab None admin] PATH_INFO: `/v3/roles/member` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:20.400765 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879e93c4-89f6-4ee1-8303-3498f8d2dd99 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:20.404236 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71edbacf-7d36-45fe-8f37-9daa5c6b79ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:59:20.409740 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-71edbacf-7d36-45fe-8f37-9daa5c6b79ab None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 11:59:20.409740 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:59:20.409740 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:59:20.409740 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:59:20.409740 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:59:20.409740 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:59:20.409740 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:59:20.409740 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:59:20.409740 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:59:20.409740 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:59:20.409740 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:59:20.409740 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:59:20.409740 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:59:20.409740 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:59:20.409740 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:59:20.409740 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:59:20.410601 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:59:20.410601 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:59:20.410601 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:59:20.410601 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:59:20.410601 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:59:20.410601 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:59:20.410601 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:59:20.410601 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:59:20.410601 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:59:20.410601 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:59:20.410601 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:59:20.410601 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 11:59:20.410601 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:59:20.410601 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:59:20.410601 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:59:20.410601 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:59:20.411494 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:59:20.411494 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:59:20.411494 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:59:20.411494 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:59:20.411494 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:59:20.411494 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:59:20.411494 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:59:20.411494 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:59:20.411494 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:59:20.411494 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:59:20.411494 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 11:59:20.411494 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 11:59:20.419058 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 133/276] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 9 11:59:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:59:20.420240 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71edbacf-7d36-45fe-8f37-9daa5c6b79ab None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:20.521949 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879e93c4-89f6-4ee1-8303-3498f8d2dd99 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:20.545233 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71edbacf-7d36-45fe-8f37-9daa5c6b79ab None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:20.559720 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4447f1f5-d51c-4f2a-baa3-e8ca223c1ccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:20.560662 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4447f1f5-d51c-4f2a-baa3-e8ca223c1ccc None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:20.561131 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4447f1f5-d51c-4f2a-baa3-e8ca223c1ccc None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:20.561731 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4447f1f5-d51c-4f2a-baa3-e8ca223c1ccc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:20.569295 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:20.574656 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4447f1f5-d51c-4f2a-baa3-e8ca223c1ccc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:59:20.579024 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf1be72-f0f5-4926-9b8d-c7096bd00ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:20.579637 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf1be72-f0f5-4926-9b8d-c7096bd00ceb None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:20.579917 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf1be72-f0f5-4926-9b8d-c7096bd00ceb None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:20.580190 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf1be72-f0f5-4926-9b8d-c7096bd00ceb None admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:20.580670 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-4447f1f5-d51c-4f2a-baa3-e8ca223c1ccc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 11:59:20.580670 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:59:20.580670 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:59:20.580670 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:59:20.580670 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:59:20.580670 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:59:20.580670 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:59:20.580670 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:59:20.580670 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:59:20.580670 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:59:20.580670 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:59:20.580670 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:59:20.580670 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:59:20.580670 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:59:20.580670 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:59:20.580670 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:59:20.581497 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:59:20.581497 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:59:20.581497 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:59:20.581497 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:59:20.581497 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:59:20.581497 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:59:20.581497 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:59:20.581497 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:59:20.581497 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:59:20.581497 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:59:20.581497 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:59:20.581497 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 11:59:20.581497 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:59:20.581497 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:59:20.581497 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:59:20.581497 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:59:20.582262 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:59:20.582262 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:59:20.582262 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:59:20.582262 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:59:20.582262 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:59:20.582262 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:59:20.582262 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:59:20.582262 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:59:20.582262 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:59:20.582262 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:59:20.582262 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 11:59:20.582262 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 11:59:20.582986 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf1be72-f0f5-4926-9b8d-c7096bd00ceb None admin] RBAC: Adding query filter params (name=member) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:59:20.582986 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf1be72-f0f5-4926-9b8d-c7096bd00ceb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:20.582986 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf1be72-f0f5-4926-9b8d-c7096bd00ceb None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:20.584570 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 144/277] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 9 11:59:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:59:20.585401 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4447f1f5-d51c-4f2a-baa3-e8ca223c1ccc None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:20.595663 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 134/278] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Sep 9 11:59:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:20.596531 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf1be72-f0f5-4926-9b8d-c7096bd00ceb None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:20.653223 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:20.705326 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4447f1f5-d51c-4f2a-baa3-e8ca223c1ccc None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:20.722181 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf1be72-f0f5-4926-9b8d-c7096bd00ceb None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:20.740786 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39b94fd-49a9-4bd0-b025-5ed3b9669a8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:20.741413 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39b94fd-49a9-4bd0-b025-5ed3b9669a8f None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:20.741686 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39b94fd-49a9-4bd0-b025-5ed3b9669a8f None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:20.741950 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39b94fd-49a9-4bd0-b025-5ed3b9669a8f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:20.743076 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39b94fd-49a9-4bd0-b025-5ed3b9669a8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:20.744534 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39b94fd-49a9-4bd0-b025-5ed3b9669a8f None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:20.746092 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 145/279] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 11:59:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:20.746866 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39b94fd-49a9-4bd0-b025-5ed3b9669a8f None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:20.760949 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d0a1e2-8b9f-4a30-bfbb-94dc897740b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:20.761555 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d0a1e2-8b9f-4a30-bfbb-94dc897740b2 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:20.761835 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d0a1e2-8b9f-4a30-bfbb-94dc897740b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:20.762109 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d0a1e2-8b9f-4a30-bfbb-94dc897740b2 None admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:20.762769 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d0a1e2-8b9f-4a30-bfbb-94dc897740b2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:59:20.763105 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d0a1e2-8b9f-4a30-bfbb-94dc897740b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:20.764490 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d0a1e2-8b9f-4a30-bfbb-94dc897740b2 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:20.773011 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 135/280] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Sep 9 11:59:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:20.776157 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ddd812-db6b-4ec6-8142-31c1f574b438 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:20.776754 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ddd812-db6b-4ec6-8142-31c1f574b438 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:20.777018 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ddd812-db6b-4ec6-8142-31c1f574b438 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:20.778591 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 136/281] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 11:59:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:59:20.779308 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ddd812-db6b-4ec6-8142-31c1f574b438 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:20.813465 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:20.847771 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39b94fd-49a9-4bd0-b025-5ed3b9669a8f None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:20.877267 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:20.881429 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ddd812-db6b-4ec6-8142-31c1f574b438 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:20.882457 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85707af-4176-446a-a6db-6fe578c9e626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:20.883115 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85707af-4176-446a-a6db-6fe578c9e626 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:20.883386 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85707af-4176-446a-a6db-6fe578c9e626 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:20.883651 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85707af-4176-446a-a6db-6fe578c9e626 None admin] PATH_INFO: `/v3/users/64a908f5ee454c1da0c477c831b0da8f` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:20.896983 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85707af-4176-446a-a6db-6fe578c9e626 None admin] RBAC: Authorizing `identity:get_user(user_id=64a908f5ee454c1da0c477c831b0da8f)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:20.898369 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85707af-4176-446a-a6db-6fe578c9e626 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:20.899998 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 146/282] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Sep 9 11:59:20 2022] GET /identity/v3/users/64a908f5ee454c1da0c477c831b0da8f => generated 293 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:20.903450 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85707af-4176-446a-a6db-6fe578c9e626 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:20.916700 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e60173-1595-4a3d-adf1-ecd558cd56ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:20.917303 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e60173-1595-4a3d-adf1-ecd558cd56ba None admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:20.917579 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e60173-1595-4a3d-adf1-ecd558cd56ba None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:20.917877 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e60173-1595-4a3d-adf1-ecd558cd56ba None admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:20.918523 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e60173-1595-4a3d-adf1-ecd558cd56ba None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:20.920012 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e60173-1595-4a3d-adf1-ecd558cd56ba None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:20.941027 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 137/283] 10.222.0.49 () {60 vars in 1132 bytes} [Fri Sep 9 11:59:20 2022] POST /identity/v3/users => generated 293 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:59:20.942205 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff32d881-fe26-4c5a-8369-f64d0fd489c3 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:20.942486 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff32d881-fe26-4c5a-8369-f64d0fd489c3 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:20.943355 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff32d881-fe26-4c5a-8369-f64d0fd489c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:20.963296 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-ff32d881-fe26-4c5a-8369-f64d0fd489c3 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:59:20.964533 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-ff32d881-fe26-4c5a-8369-f64d0fd489c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:21.032730 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85707af-4176-446a-a6db-6fe578c9e626 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:21.044700 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 138/284] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 11:59:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:59:21.045560 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff32d881-fe26-4c5a-8369-f64d0fd489c3 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:21.069011 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:21.083501 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8444b633-03ee-41cd-8799-9e84ead42d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:21.084462 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8444b633-03ee-41cd-8799-9e84ead42d7e None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:21.085173 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8444b633-03ee-41cd-8799-9e84ead42d7e None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:21.086188 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8444b633-03ee-41cd-8799-9e84ead42d7e None admin] PATH_INFO: `/v3/users/eab6a106413447d3bc8181a05ed74713` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:21.087985 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8444b633-03ee-41cd-8799-9e84ead42d7e None admin] RBAC: Authorizing `identity:get_user(user_id=eab6a106413447d3bc8181a05ed74713)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:21.089926 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8444b633-03ee-41cd-8799-9e84ead42d7e None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:21.091790 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 147/285] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Sep 9 11:59:20 2022] GET /identity/v3/users/eab6a106413447d3bc8181a05ed74713 => generated 297 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:21.100885 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8444b633-03ee-41cd-8799-9e84ead42d7e None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:21.160254 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:21.182511 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff32d881-fe26-4c5a-8369-f64d0fd489c3 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:21.230479 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36db7a5-2201-42c6-9130-1738f4d9c355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:21.231132 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36db7a5-2201-42c6-9130-1738f4d9c355 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:21.231411 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36db7a5-2201-42c6-9130-1738f4d9c355 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:21.231731 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36db7a5-2201-42c6-9130-1738f4d9c355 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:21.232460 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36db7a5-2201-42c6-9130-1738f4d9c355 None admin] RBAC: Adding query filter params (role.id=eecd2f160dd844f5906f3d345780c501, user.id=64a908f5ee454c1da0c477c831b0da8f, scope.system=all) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:59:21.232783 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36db7a5-2201-42c6-9130-1738f4d9c355 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:21.234225 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36db7a5-2201-42c6-9130-1738f4d9c355 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:21.235819 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8444b633-03ee-41cd-8799-9e84ead42d7e None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:21.246471 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 139/286] 10.222.0.49 () {56 vars in 1308 bytes} [Fri Sep 9 11:59:21 2022] GET /identity/v3/role_assignments?user.id=64a908f5ee454c1da0c477c831b0da8f&scope.system=all&role.id=eecd2f160dd844f5906f3d345780c501 => generated 226 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:21.278231 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cddc49-12f5-4c33-8d95-efe8c0990cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:21.278887 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cddc49-12f5-4c33-8d95-efe8c0990cf1 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:21.279201 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cddc49-12f5-4c33-8d95-efe8c0990cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:21.279468 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cddc49-12f5-4c33-8d95-efe8c0990cf1 None admin] PATH_INFO: `/v3/projects/5f8dc54c23f543d197444a43c6b82c01` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:21.280665 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cddc49-12f5-4c33-8d95-efe8c0990cf1 None admin] RBAC: Authorizing `identity:get_project(project_id=5f8dc54c23f543d197444a43c6b82c01)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:21.282137 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cddc49-12f5-4c33-8d95-efe8c0990cf1 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:21.284329 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 148/287] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Sep 9 11:59:21 2022] GET /identity/v3/projects/5f8dc54c23f543d197444a43c6b82c01 => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:21.291265 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36db7a5-2201-42c6-9130-1738f4d9c355 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:21.329838 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cddc49-12f5-4c33-8d95-efe8c0990cf1 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:21.405246 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:21.428865 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36db7a5-2201-42c6-9130-1738f4d9c355 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:21.453566 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cddc49-12f5-4c33-8d95-efe8c0990cf1 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:21.472429 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72174a5d-36ed-429e-9d47-50224e2a4f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:21.473106 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72174a5d-36ed-429e-9d47-50224e2a4f72 None admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:21.473382 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72174a5d-36ed-429e-9d47-50224e2a4f72 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:21.473650 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72174a5d-36ed-429e-9d47-50224e2a4f72 None admin] PATH_INFO: `/v3/projects/5f8dc54c23f543d197444a43c6b82c01/users/eab6a106413447d3bc8181a05ed74713/roles/b1e7a1e6ddc54da7ba995fb55fd93016` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:21.476204 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72174a5d-36ed-429e-9d47-50224e2a4f72 None admin] RBAC: Authorizing `identity:create_grant(project_id=5f8dc54c23f543d197444a43c6b82c01, user_id=eab6a106413447d3bc8181a05ed74713, role_id=b1e7a1e6ddc54da7ba995fb55fd93016)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:21.477657 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72174a5d-36ed-429e-9d47-50224e2a4f72 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:21.496672 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 140/288] 10.222.0.49 () {58 vars in 1439 bytes} [Fri Sep 9 11:59:21 2022] PUT /identity/v3/projects/5f8dc54c23f543d197444a43c6b82c01/users/eab6a106413447d3bc8181a05ed74713/roles/b1e7a1e6ddc54da7ba995fb55fd93016 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 11:59:21.532806 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be96033-a679-46a9-9152-393f0fdb5a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:21.533737 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be96033-a679-46a9-9152-393f0fdb5a79 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:21.534153 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be96033-a679-46a9-9152-393f0fdb5a79 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:21.534570 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be96033-a679-46a9-9152-393f0fdb5a79 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:21.559879 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be96033-a679-46a9-9152-393f0fdb5a79 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:59:21.565257 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-9be96033-a679-46a9-9152-393f0fdb5a79 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 11:59:21.565257 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:59:21.565257 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:59:21.565257 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:59:21.565257 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:59:21.565257 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:59:21.565257 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:59:21.565257 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:59:21.565257 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:59:21.565257 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:59:21.565257 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:59:21.565257 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:59:21.565257 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:59:21.565257 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:59:21.565257 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:59:21.565257 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:59:21.566328 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:59:21.566328 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:59:21.566328 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:59:21.566328 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:59:21.566328 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:59:21.566328 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:59:21.566328 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:59:21.566328 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:59:21.566328 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:59:21.566328 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:59:21.566328 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:59:21.566328 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 11:59:21.566328 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:59:21.566328 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:59:21.566328 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:59:21.566328 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:59:21.567334 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:59:21.567334 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:59:21.567334 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:59:21.567334 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:59:21.567334 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:59:21.567334 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:59:21.567334 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:59:21.567334 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:59:21.567334 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:59:21.567334 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:59:21.567334 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 11:59:21.567334 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 11:59:21.570354 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 149/289] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 9 11:59:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:59:21.577252 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72174a5d-36ed-429e-9d47-50224e2a4f72 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:21.597397 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:21.718414 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72174a5d-36ed-429e-9d47-50224e2a4f72 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:21.772671 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365c4328-aee9-4e96-8a57-990fcd609596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:21.773297 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365c4328-aee9-4e96-8a57-990fcd609596 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:21.773573 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365c4328-aee9-4e96-8a57-990fcd609596 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:21.773837 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365c4328-aee9-4e96-8a57-990fcd609596 None admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:21.774517 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365c4328-aee9-4e96-8a57-990fcd609596 None admin] RBAC: Adding query filter params (name=member) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:59:21.774817 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365c4328-aee9-4e96-8a57-990fcd609596 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:21.776222 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365c4328-aee9-4e96-8a57-990fcd609596 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:21.792320 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 141/290] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Sep 9 11:59:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:21.798117 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be96033-a679-46a9-9152-393f0fdb5a79 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:21.910457 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be96033-a679-46a9-9152-393f0fdb5a79 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:21.945015 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:21.964705 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5b1445-1ddc-4258-adfb-a9eb361b2176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:21.965342 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5b1445-1ddc-4258-adfb-a9eb361b2176 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:21.965620 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5b1445-1ddc-4258-adfb-a9eb361b2176 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:21.965890 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5b1445-1ddc-4258-adfb-a9eb361b2176 None admin] PATH_INFO: `/v3/users/976211872e2f4f188fa5e2eb48140809` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:21.971652 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5b1445-1ddc-4258-adfb-a9eb361b2176 None admin] RBAC: Authorizing `identity:get_user(user_id=976211872e2f4f188fa5e2eb48140809)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:21.973097 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5b1445-1ddc-4258-adfb-a9eb361b2176 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:21.975447 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 150/291] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Sep 9 11:59:21 2022] GET /identity/v3/users/976211872e2f4f188fa5e2eb48140809 => generated 297 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:21.980879 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365c4328-aee9-4e96-8a57-990fcd609596 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:22.076441 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:22.108834 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365c4328-aee9-4e96-8a57-990fcd609596 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:22.144127 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecb7e4f-a785-44c9-912d-4a66750beb60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:22.144741 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecb7e4f-a785-44c9-912d-4a66750beb60 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:22.145014 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecb7e4f-a785-44c9-912d-4a66750beb60 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:22.145281 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecb7e4f-a785-44c9-912d-4a66750beb60 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:22.225603 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecb7e4f-a785-44c9-912d-4a66750beb60 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:22.227071 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecb7e4f-a785-44c9-912d-4a66750beb60 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:22.324416 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:22.356744 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 142/292] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 9 11:59:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 11:59:22.362063 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5b1445-1ddc-4258-adfb-a9eb361b2176 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:22.529673 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5b1445-1ddc-4258-adfb-a9eb361b2176 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:22.577445 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb2dcf6-2115-4d90-85ec-3b310d75e7cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:22.578081 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb2dcf6-2115-4d90-85ec-3b310d75e7cf None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:22.578356 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb2dcf6-2115-4d90-85ec-3b310d75e7cf None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:22.578627 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb2dcf6-2115-4d90-85ec-3b310d75e7cf None admin] PATH_INFO: `/v3/projects/5f8dc54c23f543d197444a43c6b82c01` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:22.579804 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb2dcf6-2115-4d90-85ec-3b310d75e7cf None admin] RBAC: Authorizing `identity:get_project(project_id=5f8dc54c23f543d197444a43c6b82c01)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:22.581230 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb2dcf6-2115-4d90-85ec-3b310d75e7cf None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:22.583846 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 151/293] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Sep 9 11:59:22 2022] GET /identity/v3/projects/5f8dc54c23f543d197444a43c6b82c01 => generated 302 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:22.589712 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecb7e4f-a785-44c9-912d-4a66750beb60 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:22.735610 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecb7e4f-a785-44c9-912d-4a66750beb60 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:22.780649 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75761a4b-8b24-4810-b069-c7bc0b2f4900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:22.781567 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75761a4b-8b24-4810-b069-c7bc0b2f4900 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:22.782039 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75761a4b-8b24-4810-b069-c7bc0b2f4900 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:22.782497 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75761a4b-8b24-4810-b069-c7bc0b2f4900 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:22.783446 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75761a4b-8b24-4810-b069-c7bc0b2f4900 None admin] RBAC: Adding query filter params (role.id=eecd2f160dd844f5906f3d345780c501, scope.project.id=5f8dc54c23f543d197444a43c6b82c01, user.id=976211872e2f4f188fa5e2eb48140809) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:59:22.783982 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75761a4b-8b24-4810-b069-c7bc0b2f4900 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:22.785617 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75761a4b-8b24-4810-b069-c7bc0b2f4900 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:22.796361 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 143/294] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Sep 9 11:59:22 2022] GET /identity/v3/role_assignments?user.id=976211872e2f4f188fa5e2eb48140809&scope.project.id=5f8dc54c23f543d197444a43c6b82c01&role.id=eecd2f160dd844f5906f3d345780c501 => generated 608 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:23.664030 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5260c928-af51-4f3d-87b5-379979076161 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:23.664887 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5260c928-af51-4f3d-87b5-379979076161 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:23.665301 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5260c928-af51-4f3d-87b5-379979076161 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:23.666788 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 152/295] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 11:59:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:59:23.674749 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a885ab-3ed5-4fc3-8479-a14560744a99 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:23.675673 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a885ab-3ed5-4fc3-8479-a14560744a99 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:23.676174 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a885ab-3ed5-4fc3-8479-a14560744a99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:23.697401 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-23a885ab-3ed5-4fc3-8479-a14560744a99 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:59:23.698942 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-23a885ab-3ed5-4fc3-8479-a14560744a99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:23.732994 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:23.778746 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 144/296] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 11:59:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:59:23.845164 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf5d1b5-6a1f-4b49-9f39-39189fce3e0c None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:23.845809 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf5d1b5-6a1f-4b49-9f39-39189fce3e0c None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:23.846290 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf5d1b5-6a1f-4b49-9f39-39189fce3e0c None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:23.848096 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 153/297] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 11:59:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:59:23.854377 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad17a45-a218-4982-918f-d80ffd83bed6 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:23.854933 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad17a45-a218-4982-918f-d80ffd83bed6 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:23.855439 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad17a45-a218-4982-918f-d80ffd83bed6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:23.874317 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-1ad17a45-a218-4982-918f-d80ffd83bed6 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:59:23.875814 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-1ad17a45-a218-4982-918f-d80ffd83bed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:23.947691 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 145/298] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 11:59:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:59:24.053219 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf5d1b5-6a1f-4b49-9f39-39189fce3e0c None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:24.096363 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c564ac68-d05b-4e51-a58b-9cf56e615f09 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:24.097008 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c564ac68-d05b-4e51-a58b-9cf56e615f09 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:24.097510 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c564ac68-d05b-4e51-a58b-9cf56e615f09 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:24.100799 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 146/299] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 11:59:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 11:59:24.105949 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b4124e-c7dc-42ae-a77d-0b4cff8c4c2e None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:24.106516 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b4124e-c7dc-42ae-a77d-0b4cff8c4c2e None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:24.107017 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b4124e-c7dc-42ae-a77d-0b4cff8c4c2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:24.119092 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:24.127491 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-70b4124e-c7dc-42ae-a77d-0b4cff8c4c2e None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:59:24.128916 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-70b4124e-c7dc-42ae-a77d-0b4cff8c4c2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:24.164364 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf5d1b5-6a1f-4b49-9f39-39189fce3e0c None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:24.198624 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 147/300] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 11:59:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 11:59:24.199660 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b46b504-7fa7-49eb-ac5b-1203583c09d4 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:24.199941 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b46b504-7fa7-49eb-ac5b-1203583c09d4 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:24.200207 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b46b504-7fa7-49eb-ac5b-1203583c09d4 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:24.201569 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 148/301] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 11:59:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:59:24.203336 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23b7a48-b838-4bff-abe7-b81d25aab1cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:24.203979 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23b7a48-b838-4bff-abe7-b81d25aab1cd None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:24.204258 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23b7a48-b838-4bff-abe7-b81d25aab1cd None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:24.204527 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23b7a48-b838-4bff-abe7-b81d25aab1cd None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:24.208259 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8579422e-044e-436f-8bb2-cc5c736cf6ca None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:24.208657 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8579422e-044e-436f-8bb2-cc5c736cf6ca None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:24.209148 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8579422e-044e-436f-8bb2-cc5c736cf6ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:24.220299 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23b7a48-b838-4bff-abe7-b81d25aab1cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:59:24.226666 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-e23b7a48-b838-4bff-abe7-b81d25aab1cd None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 11:59:24.226666 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:59:24.226666 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:59:24.226666 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:59:24.226666 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:59:24.226666 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:59:24.226666 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:59:24.226666 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:59:24.226666 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:59:24.226666 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:59:24.226666 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:59:24.226666 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:59:24.226666 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:59:24.226666 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:59:24.226666 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:59:24.226666 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:59:24.227709 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:59:24.227709 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:59:24.227709 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:59:24.227709 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:59:24.227709 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:59:24.227709 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:59:24.227709 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:59:24.227709 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:59:24.227709 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:59:24.227709 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:59:24.227709 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:59:24.227709 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 11:59:24.227709 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:59:24.227709 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:59:24.227709 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:59:24.227709 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:59:24.228486 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:59:24.228486 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:59:24.228486 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:59:24.228486 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:59:24.228486 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:59:24.228486 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:59:24.228486 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:59:24.228486 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:59:24.228486 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:59:24.228486 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:59:24.228486 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 11:59:24.228486 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 11:59:24.229101 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-8579422e-044e-436f-8bb2-cc5c736cf6ca None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:59:24.229171 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 154/302] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 9 11:59:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:59:24.229171 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23b7a48-b838-4bff-abe7-b81d25aab1cd None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:24.229286 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-8579422e-044e-436f-8bb2-cc5c736cf6ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:24.252295 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:24.305716 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 149/303] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 11:59:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:59:24.311799 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8579422e-044e-436f-8bb2-cc5c736cf6ca None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:24.349297 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23b7a48-b838-4bff-abe7-b81d25aab1cd None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:24.369484 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:24.392517 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efd3eb5-9bee-41cd-a59e-e34edffbb326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:24.393154 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efd3eb5-9bee-41cd-a59e-e34edffbb326 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:24.393587 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efd3eb5-9bee-41cd-a59e-e34edffbb326 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:24.393790 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efd3eb5-9bee-41cd-a59e-e34edffbb326 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:24.395088 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efd3eb5-9bee-41cd-a59e-e34edffbb326 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:24.397281 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efd3eb5-9bee-41cd-a59e-e34edffbb326 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:24.398886 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 155/304] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 11:59:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:24.404755 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efd3eb5-9bee-41cd-a59e-e34edffbb326 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:24.449561 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8579422e-044e-436f-8bb2-cc5c736cf6ca None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:24.492924 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7106ff98-a7be-4018-bad2-55b11c477d81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:24.493535 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7106ff98-a7be-4018-bad2-55b11c477d81 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:24.493811 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7106ff98-a7be-4018-bad2-55b11c477d81 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:24.494082 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7106ff98-a7be-4018-bad2-55b11c477d81 None admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:24.494743 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7106ff98-a7be-4018-bad2-55b11c477d81 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:59:24.495072 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7106ff98-a7be-4018-bad2-55b11c477d81 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:24.496456 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7106ff98-a7be-4018-bad2-55b11c477d81 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:24.509476 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 150/305] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Sep 9 11:59:24 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:24.509476 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7106ff98-a7be-4018-bad2-55b11c477d81 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:24.515689 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efd3eb5-9bee-41cd-a59e-e34edffbb326 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:24.524848 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:24.551787 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fcfe8b-5001-492e-84c6-8364834d522e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:24.553062 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fcfe8b-5001-492e-84c6-8364834d522e None admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:24.553344 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fcfe8b-5001-492e-84c6-8364834d522e None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:24.553618 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fcfe8b-5001-492e-84c6-8364834d522e None admin] PATH_INFO: `/v3/groups` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:24.554248 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fcfe8b-5001-492e-84c6-8364834d522e None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:24.555722 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fcfe8b-5001-492e-84c6-8364834d522e None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:24.573667 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 156/306] 10.222.0.49 () {60 vars in 1134 bytes} [Fri Sep 9 11:59:24 2022] POST /identity/v3/groups => generated 229 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:59:24.574802 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fcfe8b-5001-492e-84c6-8364834d522e None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:24.618633 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7106ff98-a7be-4018-bad2-55b11c477d81 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:24.645353 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:24.667187 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ecb474-88cc-4a6d-bd1a-4463f3702302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:24.667842 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ecb474-88cc-4a6d-bd1a-4463f3702302 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:24.668124 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ecb474-88cc-4a6d-bd1a-4463f3702302 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:24.668398 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ecb474-88cc-4a6d-bd1a-4463f3702302 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:24.680748 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ecb474-88cc-4a6d-bd1a-4463f3702302 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:59:24.686637 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-83ecb474-88cc-4a6d-bd1a-4463f3702302 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 11:59:24.686637 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:59:24.686637 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:59:24.686637 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:59:24.686637 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:59:24.686637 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:59:24.686637 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:59:24.686637 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:59:24.686637 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:59:24.686637 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:59:24.686637 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:59:24.686637 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:59:24.686637 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:59:24.686637 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:59:24.686637 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:59:24.686637 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:59:24.687793 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:59:24.687793 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:59:24.687793 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:59:24.687793 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:59:24.687793 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:59:24.687793 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:59:24.687793 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:59:24.687793 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:59:24.687793 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:59:24.687793 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:59:24.687793 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:59:24.687793 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 11:59:24.687793 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:59:24.687793 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:59:24.687793 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:59:24.687793 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:59:24.688596 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:59:24.688596 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:59:24.688596 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:59:24.688596 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:59:24.688596 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:59:24.688596 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:59:24.688596 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:59:24.688596 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:59:24.688596 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:59:24.688596 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:59:24.688596 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 11:59:24.688596 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 11:59:24.688596 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 151/307] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 9 11:59:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:59:24.689507 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ecb474-88cc-4a6d-bd1a-4463f3702302 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:24.714771 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fcfe8b-5001-492e-84c6-8364834d522e None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:24.757138 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697f45a2-18ab-4ba7-bbca-0b471ec5b855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:24.757438 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697f45a2-18ab-4ba7-bbca-0b471ec5b855 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:24.757711 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697f45a2-18ab-4ba7-bbca-0b471ec5b855 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:24.757980 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697f45a2-18ab-4ba7-bbca-0b471ec5b855 None admin] PATH_INFO: `/v3/users/3527c7a46a3148b882c3511c9febe789` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:24.775855 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697f45a2-18ab-4ba7-bbca-0b471ec5b855 None admin] RBAC: Authorizing `identity:get_user(user_id=3527c7a46a3148b882c3511c9febe789)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:24.777420 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697f45a2-18ab-4ba7-bbca-0b471ec5b855 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:24.779305 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 157/308] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Sep 9 11:59:24 2022] GET /identity/v3/users/3527c7a46a3148b882c3511c9febe789 => generated 293 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:24.779911 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697f45a2-18ab-4ba7-bbca-0b471ec5b855 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:24.782637 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:24.809316 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ecb474-88cc-4a6d-bd1a-4463f3702302 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:24.844588 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8445fd33-9aeb-4523-aeb5-b4e20156b9a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:24.845211 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8445fd33-9aeb-4523-aeb5-b4e20156b9a3 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:24.845485 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8445fd33-9aeb-4523-aeb5-b4e20156b9a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:24.845747 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8445fd33-9aeb-4523-aeb5-b4e20156b9a3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:24.859670 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8445fd33-9aeb-4523-aeb5-b4e20156b9a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:59:24.866189 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-8445fd33-9aeb-4523-aeb5-b4e20156b9a3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 11:59:24.866189 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:59:24.866189 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:59:24.866189 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:59:24.866189 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:59:24.866189 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:59:24.866189 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:59:24.866189 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:59:24.866189 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:59:24.866189 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:59:24.866189 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:59:24.866189 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:59:24.866189 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:59:24.866189 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:59:24.866189 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:59:24.866189 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:59:24.867285 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:59:24.867285 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:59:24.867285 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:59:24.867285 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:59:24.867285 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:59:24.867285 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:59:24.867285 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:59:24.867285 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:59:24.867285 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:59:24.867285 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:59:24.867285 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:59:24.867285 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 11:59:24.867285 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:59:24.867285 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:59:24.867285 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:59:24.867285 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:59:24.871978 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:59:24.871978 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:59:24.871978 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:59:24.871978 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:59:24.871978 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:59:24.871978 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:59:24.871978 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:59:24.871978 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:59:24.871978 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:59:24.871978 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:59:24.871978 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 11:59:24.871978 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 11:59:24.871978 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 152/309] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 9 11:59:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:59:24.871978 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8445fd33-9aeb-4523-aeb5-b4e20156b9a3 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:24.899577 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697f45a2-18ab-4ba7-bbca-0b471ec5b855 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:24.901857 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:24.941047 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2509ef-b7e4-43f5-9999-833859f1ada9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:24.941754 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2509ef-b7e4-43f5-9999-833859f1ada9 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:24.942030 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2509ef-b7e4-43f5-9999-833859f1ada9 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:24.942298 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2509ef-b7e4-43f5-9999-833859f1ada9 None admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:24.943598 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2509ef-b7e4-43f5-9999-833859f1ada9 None admin] RBAC: Adding query filter params (name=member) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:59:24.943893 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2509ef-b7e4-43f5-9999-833859f1ada9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:24.945260 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2509ef-b7e4-43f5-9999-833859f1ada9 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:24.954670 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 158/310] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Sep 9 11:59:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:24.956151 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2509ef-b7e4-43f5-9999-833859f1ada9 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:24.983601 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8445fd33-9aeb-4523-aeb5-b4e20156b9a3 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:25.018914 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63080b4c-7e67-4f5f-99aa-379a597d6644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:25.019572 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63080b4c-7e67-4f5f-99aa-379a597d6644 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:25.019850 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63080b4c-7e67-4f5f-99aa-379a597d6644 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:25.020122 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63080b4c-7e67-4f5f-99aa-379a597d6644 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:25.020863 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63080b4c-7e67-4f5f-99aa-379a597d6644 None admin] RBAC: Adding query filter params (role.id=b1e7a1e6ddc54da7ba995fb55fd93016, user.id=3527c7a46a3148b882c3511c9febe789, scope.system=all) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:59:25.021167 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63080b4c-7e67-4f5f-99aa-379a597d6644 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:25.022602 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63080b4c-7e67-4f5f-99aa-379a597d6644 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:25.034937 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 153/311] 10.222.0.49 () {56 vars in 1308 bytes} [Fri Sep 9 11:59:24 2022] GET /identity/v3/role_assignments?user.id=3527c7a46a3148b882c3511c9febe789&scope.system=all&role.id=b1e7a1e6ddc54da7ba995fb55fd93016 => generated 226 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:25.034937 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63080b4c-7e67-4f5f-99aa-379a597d6644 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:25.034937 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:25.079755 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2509ef-b7e4-43f5-9999-833859f1ada9 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:25.122802 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d372cd-6589-4f85-bc1c-278db6542a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:25.123589 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d372cd-6589-4f85-bc1c-278db6542a33 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:25.123965 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d372cd-6589-4f85-bc1c-278db6542a33 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:25.124333 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d372cd-6589-4f85-bc1c-278db6542a33 None admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:25.125105 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d372cd-6589-4f85-bc1c-278db6542a33 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:59:25.125504 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d372cd-6589-4f85-bc1c-278db6542a33 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:25.127052 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d372cd-6589-4f85-bc1c-278db6542a33 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:25.137948 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 159/312] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Sep 9 11:59:24 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:25.143999 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d372cd-6589-4f85-bc1c-278db6542a33 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:25.163424 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63080b4c-7e67-4f5f-99aa-379a597d6644 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:25.187692 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:25.220389 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b25d011-396c-4b0d-983e-1311cb9ffe15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:25.221899 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b25d011-396c-4b0d-983e-1311cb9ffe15 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:25.222592 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b25d011-396c-4b0d-983e-1311cb9ffe15 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:25.223033 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b25d011-396c-4b0d-983e-1311cb9ffe15 None admin] PATH_INFO: `/v3/users/64a908f5ee454c1da0c477c831b0da8f` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:25.224712 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b25d011-396c-4b0d-983e-1311cb9ffe15 None admin] RBAC: Authorizing `identity:get_user(user_id=64a908f5ee454c1da0c477c831b0da8f)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:25.226765 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b25d011-396c-4b0d-983e-1311cb9ffe15 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:25.228586 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 154/313] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Sep 9 11:59:25 2022] GET /identity/v3/users/64a908f5ee454c1da0c477c831b0da8f => generated 293 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:25.234490 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b25d011-396c-4b0d-983e-1311cb9ffe15 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:25.293588 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d372cd-6589-4f85-bc1c-278db6542a33 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:25.314113 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:25.334623 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b827108-974c-4e7a-8c03-ea96ff542388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:25.335372 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b827108-974c-4e7a-8c03-ea96ff542388 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:25.335720 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b827108-974c-4e7a-8c03-ea96ff542388 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:25.336062 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b827108-974c-4e7a-8c03-ea96ff542388 None admin] PATH_INFO: `/v3/users/eab6a106413447d3bc8181a05ed74713` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:25.337375 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b827108-974c-4e7a-8c03-ea96ff542388 None admin] RBAC: Authorizing `identity:get_user(user_id=eab6a106413447d3bc8181a05ed74713)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:25.338832 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b827108-974c-4e7a-8c03-ea96ff542388 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:25.341649 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 160/314] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Sep 9 11:59:25 2022] GET /identity/v3/users/eab6a106413447d3bc8181a05ed74713 => generated 297 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:59:25.347238 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b827108-974c-4e7a-8c03-ea96ff542388 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:25.349953 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b25d011-396c-4b0d-983e-1311cb9ffe15 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:25.391385 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdbb2f9-52ad-41b8-a0a4-b8d90b4883de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:25.392040 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdbb2f9-52ad-41b8-a0a4-b8d90b4883de None admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:25.392318 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdbb2f9-52ad-41b8-a0a4-b8d90b4883de None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:25.392649 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdbb2f9-52ad-41b8-a0a4-b8d90b4883de None admin] PATH_INFO: `/v3/system/users/64a908f5ee454c1da0c477c831b0da8f/roles/eecd2f160dd844f5906f3d345780c501` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:25.394309 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdbb2f9-52ad-41b8-a0a4-b8d90b4883de None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=64a908f5ee454c1da0c477c831b0da8f, role_id=eecd2f160dd844f5906f3d345780c501)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:25.396566 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdbb2f9-52ad-41b8-a0a4-b8d90b4883de None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:25.410075 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 155/315] 10.222.0.49 () {58 vars in 1334 bytes} [Fri Sep 9 11:59:25 2022] PUT /identity/v3/system/users/64a908f5ee454c1da0c477c831b0da8f/roles/eecd2f160dd844f5906f3d345780c501 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 11:59:25.463068 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:25.470695 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b827108-974c-4e7a-8c03-ea96ff542388 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:25.513097 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db44f1a-a93f-4acb-ae2b-3fb830852299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:25.513783 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db44f1a-a93f-4acb-ae2b-3fb830852299 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:25.514117 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db44f1a-a93f-4acb-ae2b-3fb830852299 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:25.514389 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db44f1a-a93f-4acb-ae2b-3fb830852299 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:25.597969 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db44f1a-a93f-4acb-ae2b-3fb830852299 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:25.600340 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db44f1a-a93f-4acb-ae2b-3fb830852299 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:25.705382 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:25.714439 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 161/316] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 9 11:59:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 11:59:25.720028 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdbb2f9-52ad-41b8-a0a4-b8d90b4883de None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:25.825934 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdbb2f9-52ad-41b8-a0a4-b8d90b4883de None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:25.861451 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3263cf-897b-45e9-905d-3bbce679eeea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:25.862092 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3263cf-897b-45e9-905d-3bbce679eeea None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:25.862370 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3263cf-897b-45e9-905d-3bbce679eeea None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:25.862640 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3263cf-897b-45e9-905d-3bbce679eeea None admin] PATH_INFO: `/v3/projects/5f8dc54c23f543d197444a43c6b82c01` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:25.863800 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3263cf-897b-45e9-905d-3bbce679eeea None admin] RBAC: Authorizing `identity:get_project(project_id=5f8dc54c23f543d197444a43c6b82c01)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:25.865936 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3263cf-897b-45e9-905d-3bbce679eeea None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:25.867988 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 156/317] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Sep 9 11:59:25 2022] GET /identity/v3/projects/5f8dc54c23f543d197444a43c6b82c01 => generated 302 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:25.873923 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db44f1a-a93f-4acb-ae2b-3fb830852299 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:25.980624 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db44f1a-a93f-4acb-ae2b-3fb830852299 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:26.016882 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff03640e-e075-4d58-b88f-899af66ab12d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:26.017505 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff03640e-e075-4d58-b88f-899af66ab12d None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:26.017779 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff03640e-e075-4d58-b88f-899af66ab12d None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:26.018050 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff03640e-e075-4d58-b88f-899af66ab12d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:26.018782 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff03640e-e075-4d58-b88f-899af66ab12d None admin] RBAC: Adding query filter params (role.id=b1e7a1e6ddc54da7ba995fb55fd93016, scope.project.id=5f8dc54c23f543d197444a43c6b82c01, user.id=eab6a106413447d3bc8181a05ed74713) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:59:26.019121 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff03640e-e075-4d58-b88f-899af66ab12d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:26.020556 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff03640e-e075-4d58-b88f-899af66ab12d None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:26.028394 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 162/318] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Sep 9 11:59:25 2022] GET /identity/v3/role_assignments?user.id=eab6a106413447d3bc8181a05ed74713&scope.project.id=5f8dc54c23f543d197444a43c6b82c01&role.id=b1e7a1e6ddc54da7ba995fb55fd93016 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:26.845003 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8756027c-dbdd-411c-99f0-59d92dc23009 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:26.845529 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8756027c-dbdd-411c-99f0-59d92dc23009 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:26.845529 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8756027c-dbdd-411c-99f0-59d92dc23009 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:26.847065 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 157/319] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 11:59:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:59:26.852843 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f71ce87-3a3e-4729-8c90-bb44ec264277 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:26.853137 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f71ce87-3a3e-4729-8c90-bb44ec264277 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:26.853410 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f71ce87-3a3e-4729-8c90-bb44ec264277 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:26.872165 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-1f71ce87-3a3e-4729-8c90-bb44ec264277 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:59:26.873358 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-1f71ce87-3a3e-4729-8c90-bb44ec264277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:26.929275 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:26.944599 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 163/320] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 11:59:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:59:27.213299 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8756027c-dbdd-411c-99f0-59d92dc23009 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:27.216332 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:27.325953 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e586ffe5-c4c3-4c88-a86f-d659e4a97001 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:27.326238 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e586ffe5-c4c3-4c88-a86f-d659e4a97001 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:27.326519 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e586ffe5-c4c3-4c88-a86f-d659e4a97001 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:27.327914 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 164/321] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 11:59:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 11:59:27.333044 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f87178-13a4-4da4-a5b5-c7cee37ccbb6 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:27.333940 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f87178-13a4-4da4-a5b5-c7cee37ccbb6 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:27.334485 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f87178-13a4-4da4-a5b5-c7cee37ccbb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:27.337863 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8756027c-dbdd-411c-99f0-59d92dc23009 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:27.356522 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-86f87178-13a4-4da4-a5b5-c7cee37ccbb6 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:59:27.358123 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-86f87178-13a4-4da4-a5b5-c7cee37ccbb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:27.374579 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f47705e-a32c-4d17-a67e-1fdc65a4c330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:27.375308 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f47705e-a32c-4d17-a67e-1fdc65a4c330 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:27.375706 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f47705e-a32c-4d17-a67e-1fdc65a4c330 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:27.375977 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f47705e-a32c-4d17-a67e-1fdc65a4c330 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:27.377078 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f47705e-a32c-4d17-a67e-1fdc65a4c330 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:27.378525 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f47705e-a32c-4d17-a67e-1fdc65a4c330 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:27.380990 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 158/322] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 11:59:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:27.386252 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f47705e-a32c-4d17-a67e-1fdc65a4c330 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:27.434100 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 165/323] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 11:59:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:59:27.464220 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:27.499153 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f47705e-a32c-4d17-a67e-1fdc65a4c330 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:27.536947 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1defe782-4d73-45a4-9468-565fd071823f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:27.537558 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1defe782-4d73-45a4-9468-565fd071823f None admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:27.537831 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1defe782-4d73-45a4-9468-565fd071823f None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:27.538091 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1defe782-4d73-45a4-9468-565fd071823f None admin] PATH_INFO: `/v3/groups` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:27.538706 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1defe782-4d73-45a4-9468-565fd071823f None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:27.540160 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1defe782-4d73-45a4-9468-565fd071823f None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:27.558178 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 159/324] 10.222.0.49 () {60 vars in 1134 bytes} [Fri Sep 9 11:59:27 2022] POST /identity/v3/groups => generated 226 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 11:59:27.611342 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d57ef7-bc3f-4a91-8984-15187ec78321 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:27.611670 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d57ef7-bc3f-4a91-8984-15187ec78321 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:27.611901 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d57ef7-bc3f-4a91-8984-15187ec78321 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:27.613493 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 166/325] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 11:59:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:59:27.619879 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e1f831-4374-4c64-beb2-2b1cb10e2ce3 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:27.620236 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e1f831-4374-4c64-beb2-2b1cb10e2ce3 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:27.620436 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e1f831-4374-4c64-beb2-2b1cb10e2ce3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:27.640170 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-73e1f831-4374-4c64-beb2-2b1cb10e2ce3 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:59:27.641416 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-73e1f831-4374-4c64-beb2-2b1cb10e2ce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:27.713634 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d57ef7-bc3f-4a91-8984-15187ec78321 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:27.717332 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 160/326] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 11:59:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:59:27.825588 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d57ef7-bc3f-4a91-8984-15187ec78321 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:27.845392 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:27.866580 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79317744-cbb2-42e7-920b-855cd0a2c6d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:27.867247 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79317744-cbb2-42e7-920b-855cd0a2c6d6 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:27.867522 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79317744-cbb2-42e7-920b-855cd0a2c6d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:27.867786 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79317744-cbb2-42e7-920b-855cd0a2c6d6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:27.879018 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79317744-cbb2-42e7-920b-855cd0a2c6d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:59:27.884094 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-79317744-cbb2-42e7-920b-855cd0a2c6d6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 11:59:27.884094 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:59:27.884094 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:59:27.884094 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:59:27.884094 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:59:27.884094 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:59:27.884094 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:59:27.884094 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:59:27.884094 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:59:27.884094 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:59:27.884094 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:59:27.884094 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:59:27.884094 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:59:27.884094 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:59:27.884094 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:59:27.884094 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:59:27.884972 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:59:27.884972 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:59:27.884972 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:59:27.884972 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:59:27.884972 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:59:27.884972 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:59:27.884972 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:59:27.884972 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:59:27.884972 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:59:27.884972 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:59:27.884972 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:59:27.884972 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 11:59:27.884972 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:59:27.884972 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:59:27.884972 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:59:27.884972 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:59:27.885797 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:59:27.885797 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:59:27.885797 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:59:27.885797 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:59:27.885797 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:59:27.885797 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:59:27.885797 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:59:27.885797 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:59:27.885797 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:59:27.885797 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:59:27.885797 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 11:59:27.885797 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 11:59:27.885797 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 167/327] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 9 11:59:27 2022] GET /identity/v3/roles/reader => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:59:27.891031 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e1f831-4374-4c64-beb2-2b1cb10e2ce3 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:27.963590 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:27.993740 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79317744-cbb2-42e7-920b-855cd0a2c6d6 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:28.008023 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e1f831-4374-4c64-beb2-2b1cb10e2ce3 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:28.054154 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e016d3-f790-4f59-b2cf-94a3a934546f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:28.055058 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e016d3-f790-4f59-b2cf-94a3a934546f None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:28.055436 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e016d3-f790-4f59-b2cf-94a3a934546f None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:28.055803 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e016d3-f790-4f59-b2cf-94a3a934546f None admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:28.056740 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e016d3-f790-4f59-b2cf-94a3a934546f None admin] RBAC: Adding query filter params (name=reader) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:59:28.057042 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e016d3-f790-4f59-b2cf-94a3a934546f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:28.058427 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e016d3-f790-4f59-b2cf-94a3a934546f None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:28.068681 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 161/328] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Sep 9 11:59:27 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:28.077795 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e016d3-f790-4f59-b2cf-94a3a934546f None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:28.134159 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79317744-cbb2-42e7-920b-855cd0a2c6d6 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:28.177521 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e483bd-2291-4b81-a57c-baa72eb2b134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:28.178167 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e483bd-2291-4b81-a57c-baa72eb2b134 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:28.178446 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e483bd-2291-4b81-a57c-baa72eb2b134 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:28.178715 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e483bd-2291-4b81-a57c-baa72eb2b134 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:28.193888 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e483bd-2291-4b81-a57c-baa72eb2b134 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:59:28.199196 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-32e483bd-2291-4b81-a57c-baa72eb2b134 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 11:59:28.199196 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:59:28.199196 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:59:28.199196 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:59:28.199196 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:59:28.199196 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:59:28.199196 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:59:28.199196 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:59:28.199196 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:59:28.199196 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:59:28.199196 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:59:28.199196 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:59:28.199196 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:59:28.199196 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:59:28.199196 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:59:28.199196 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:59:28.200067 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:59:28.200067 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:59:28.200067 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:59:28.200067 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:59:28.200067 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:59:28.200067 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:59:28.200067 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:59:28.200067 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:59:28.200067 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:59:28.200067 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:59:28.200067 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:59:28.200067 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 11:59:28.200067 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:59:28.200067 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:59:28.200067 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:59:28.200067 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:59:28.200853 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:59:28.200853 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:59:28.200853 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:59:28.200853 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:59:28.200853 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:59:28.200853 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:59:28.200853 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:59:28.200853 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:59:28.200853 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:59:28.200853 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:59:28.200853 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 11:59:28.200853 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 11:59:28.201909 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 168/329] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 9 11:59:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:59:28.203738 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e016d3-f790-4f59-b2cf-94a3a934546f None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:28.206779 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e483bd-2291-4b81-a57c-baa72eb2b134 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:28.228012 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:28.242712 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76522faf-8473-4791-a194-19246491b035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:28.243391 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76522faf-8473-4791-a194-19246491b035 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:28.243669 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76522faf-8473-4791-a194-19246491b035 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:28.243991 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76522faf-8473-4791-a194-19246491b035 None admin] PATH_INFO: `/v3/users/3527c7a46a3148b882c3511c9febe789` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:28.245250 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76522faf-8473-4791-a194-19246491b035 None admin] RBAC: Authorizing `identity:get_user(user_id=3527c7a46a3148b882c3511c9febe789)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:28.246721 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76522faf-8473-4791-a194-19246491b035 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:28.250168 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 162/330] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Sep 9 11:59:28 2022] GET /identity/v3/users/3527c7a46a3148b882c3511c9febe789 => generated 293 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:28.256753 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76522faf-8473-4791-a194-19246491b035 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:28.284256 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:28.366202 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e483bd-2291-4b81-a57c-baa72eb2b134 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:28.412137 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76522faf-8473-4791-a194-19246491b035 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:28.427519 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fa1591-5567-4429-b0aa-eb5916469cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:28.428358 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fa1591-5567-4429-b0aa-eb5916469cf3 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:28.428628 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fa1591-5567-4429-b0aa-eb5916469cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:28.428891 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fa1591-5567-4429-b0aa-eb5916469cf3 None admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:28.429564 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fa1591-5567-4429-b0aa-eb5916469cf3 None admin] RBAC: Adding query filter params (name=member) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:59:28.429863 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fa1591-5567-4429-b0aa-eb5916469cf3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:28.431262 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fa1591-5567-4429-b0aa-eb5916469cf3 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:28.453495 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 169/331] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Sep 9 11:59:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:59:28.463961 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fa1591-5567-4429-b0aa-eb5916469cf3 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:28.466859 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdd0dc9-f511-4eb2-95da-45ea5ce9e2f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:28.467657 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdd0dc9-f511-4eb2-95da-45ea5ce9e2f0 None admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:28.467943 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdd0dc9-f511-4eb2-95da-45ea5ce9e2f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:28.468222 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdd0dc9-f511-4eb2-95da-45ea5ce9e2f0 None admin] PATH_INFO: `/v3/system/users/3527c7a46a3148b882c3511c9febe789/roles/b1e7a1e6ddc54da7ba995fb55fd93016` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:28.469778 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdd0dc9-f511-4eb2-95da-45ea5ce9e2f0 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=3527c7a46a3148b882c3511c9febe789, role_id=b1e7a1e6ddc54da7ba995fb55fd93016)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:28.471344 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdd0dc9-f511-4eb2-95da-45ea5ce9e2f0 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:28.487738 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 163/332] 10.222.0.49 () {58 vars in 1334 bytes} [Fri Sep 9 11:59:28 2022] PUT /identity/v3/system/users/3527c7a46a3148b882c3511c9febe789/roles/b1e7a1e6ddc54da7ba995fb55fd93016 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:59:28.619860 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:28.623658 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fa1591-5567-4429-b0aa-eb5916469cf3 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:28.676490 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f94cee-20a6-4bb2-bc46-385347c1c56e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:28.677946 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f94cee-20a6-4bb2-bc46-385347c1c56e None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:28.678299 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f94cee-20a6-4bb2-bc46-385347c1c56e None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:28.678486 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f94cee-20a6-4bb2-bc46-385347c1c56e None admin] PATH_INFO: `/v3/users/64a908f5ee454c1da0c477c831b0da8f` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:28.683900 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f94cee-20a6-4bb2-bc46-385347c1c56e None admin] RBAC: Authorizing `identity:get_user(user_id=64a908f5ee454c1da0c477c831b0da8f)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:28.685391 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f94cee-20a6-4bb2-bc46-385347c1c56e None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:28.687467 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 170/333] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Sep 9 11:59:28 2022] GET /identity/v3/users/64a908f5ee454c1da0c477c831b0da8f => generated 293 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:28.694469 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdd0dc9-f511-4eb2-95da-45ea5ce9e2f0 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:28.790758 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:28.852803 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdd0dc9-f511-4eb2-95da-45ea5ce9e2f0 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:28.890637 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525e83ac-59db-4d99-ab82-9d3e85b3f91d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:28.891189 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525e83ac-59db-4d99-ab82-9d3e85b3f91d None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:28.891719 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525e83ac-59db-4d99-ab82-9d3e85b3f91d None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:28.891719 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525e83ac-59db-4d99-ab82-9d3e85b3f91d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:28.892428 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525e83ac-59db-4d99-ab82-9d3e85b3f91d None admin] RBAC: Adding query filter params (role.id=eecd2f160dd844f5906f3d345780c501, user.id=64a908f5ee454c1da0c477c831b0da8f, scope.system=all) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:59:28.892783 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525e83ac-59db-4d99-ab82-9d3e85b3f91d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:28.894223 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525e83ac-59db-4d99-ab82-9d3e85b3f91d None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:28.909478 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 164/334] 10.222.0.49 () {56 vars in 1308 bytes} [Fri Sep 9 11:59:28 2022] GET /identity/v3/role_assignments?user.id=64a908f5ee454c1da0c477c831b0da8f&scope.system=all&role.id=eecd2f160dd844f5906f3d345780c501 => generated 510 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:30.203511 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de5edc7-ef30-4287-a7f5-7a08ebb433dc None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:30.204034 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de5edc7-ef30-4287-a7f5-7a08ebb433dc None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:30.204034 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de5edc7-ef30-4287-a7f5-7a08ebb433dc None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:30.205665 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 171/335] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 11:59:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:59:30.212889 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64487de-5c30-4aab-9c0a-bd12e775f8e9 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:30.213349 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64487de-5c30-4aab-9c0a-bd12e775f8e9 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:30.213438 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64487de-5c30-4aab-9c0a-bd12e775f8e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:30.235497 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-e64487de-5c30-4aab-9c0a-bd12e775f8e9 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:59:30.236702 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-e64487de-5c30-4aab-9c0a-bd12e775f8e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:30.320573 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 165/336] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 11:59:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 11:59:30.820203 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72add82-861c-479e-868d-0c5da2e75841 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:30.820594 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72add82-861c-479e-868d-0c5da2e75841 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:30.820770 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72add82-861c-479e-868d-0c5da2e75841 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:30.822103 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 172/337] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 11:59:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:59:30.827642 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e930c0ba-5763-4dc0-8d49-1a92b9cf75b6 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:30.827931 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e930c0ba-5763-4dc0-8d49-1a92b9cf75b6 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:30.828206 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e930c0ba-5763-4dc0-8d49-1a92b9cf75b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:30.852957 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-e930c0ba-5763-4dc0-8d49-1a92b9cf75b6 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:59:30.854155 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-e930c0ba-5763-4dc0-8d49-1a92b9cf75b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:30.891823 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72add82-861c-479e-868d-0c5da2e75841 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:30.901144 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:30.950437 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 166/338] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 11:59:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:59:30.951892 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7445a2a4-d798-44af-ac73-60691a0b63f7 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:30.952188 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7445a2a4-d798-44af-ac73-60691a0b63f7 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:30.952462 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7445a2a4-d798-44af-ac73-60691a0b63f7 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:30.953718 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 167/339] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 11:59:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:59:30.992807 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841854e0-816d-4baf-9e8e-331a815ca788 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:30.993082 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841854e0-816d-4baf-9e8e-331a815ca788 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:30.993348 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841854e0-816d-4baf-9e8e-331a815ca788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:31.024015 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-841854e0-816d-4baf-9e8e-331a815ca788 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:59:31.025235 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-841854e0-816d-4baf-9e8e-331a815ca788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:31.075691 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72add82-861c-479e-868d-0c5da2e75841 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:31.124738 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c323fc-a94d-4d55-a488-9390acced08e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:31.125585 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c323fc-a94d-4d55-a488-9390acced08e None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:31.125861 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c323fc-a94d-4d55-a488-9390acced08e None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:31.126258 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c323fc-a94d-4d55-a488-9390acced08e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:31.139119 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c323fc-a94d-4d55-a488-9390acced08e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:59:31.145999 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-32c323fc-a94d-4d55-a488-9390acced08e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 11:59:31.145999 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:59:31.145999 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:59:31.145999 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:59:31.145999 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:59:31.145999 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:59:31.145999 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:59:31.145999 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:59:31.145999 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:59:31.145999 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:59:31.145999 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:59:31.145999 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:59:31.145999 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:59:31.145999 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:59:31.145999 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:59:31.145999 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:59:31.147089 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:59:31.147089 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:59:31.147089 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:59:31.147089 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:59:31.147089 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:59:31.147089 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:59:31.147089 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:59:31.147089 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:59:31.147089 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:59:31.147089 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:59:31.147089 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:59:31.147089 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 11:59:31.147089 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:59:31.147089 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:59:31.147089 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:59:31.147089 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:59:31.147898 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:59:31.147898 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:59:31.147898 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:59:31.147898 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:59:31.147898 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:59:31.147898 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:59:31.147898 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:59:31.147898 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:59:31.147898 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:59:31.147898 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:59:31.147898 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 11:59:31.147898 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 11:59:31.147898 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 173/340] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 9 11:59:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:59:31.162021 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c323fc-a94d-4d55-a488-9390acced08e None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:31.202040 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 168/341] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 11:59:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 11:59:31.209301 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:31.290842 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c323fc-a94d-4d55-a488-9390acced08e None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:31.331263 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254a3756-941e-4dfa-993f-1a20df7ed72d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:31.331874 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254a3756-941e-4dfa-993f-1a20df7ed72d None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:31.332202 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254a3756-941e-4dfa-993f-1a20df7ed72d None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:31.332480 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254a3756-941e-4dfa-993f-1a20df7ed72d None admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:31.333672 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254a3756-941e-4dfa-993f-1a20df7ed72d None admin] RBAC: Adding query filter params (name=member) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:59:31.333974 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254a3756-941e-4dfa-993f-1a20df7ed72d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:31.335376 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254a3756-941e-4dfa-993f-1a20df7ed72d None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:31.344495 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 174/342] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Sep 9 11:59:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:59:31.353952 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841854e0-816d-4baf-9e8e-331a815ca788 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:31.449248 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254a3756-941e-4dfa-993f-1a20df7ed72d None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:31.486814 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841854e0-816d-4baf-9e8e-331a815ca788 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:31.491212 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:31.538063 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf351f65-a0aa-4fd0-8644-2672c04e97d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:31.538679 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf351f65-a0aa-4fd0-8644-2672c04e97d8 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:31.538994 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf351f65-a0aa-4fd0-8644-2672c04e97d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:31.539266 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf351f65-a0aa-4fd0-8644-2672c04e97d8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:31.602958 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:31.615827 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254a3756-941e-4dfa-993f-1a20df7ed72d None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:31.680198 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0727be5f-03ec-421a-a43a-2a8322c87cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:31.680841 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0727be5f-03ec-421a-a43a-2a8322c87cf6 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:31.681117 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0727be5f-03ec-421a-a43a-2a8322c87cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:31.681449 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0727be5f-03ec-421a-a43a-2a8322c87cf6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:31.694864 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0727be5f-03ec-421a-a43a-2a8322c87cf6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:59:31.701199 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-0727be5f-03ec-421a-a43a-2a8322c87cf6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 11:59:31.701199 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:59:31.701199 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:59:31.701199 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:59:31.701199 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:59:31.701199 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:59:31.701199 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:59:31.701199 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:59:31.701199 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:59:31.701199 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:59:31.701199 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:59:31.701199 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:59:31.701199 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:59:31.701199 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:59:31.701199 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:59:31.701199 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:59:31.702073 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:59:31.702073 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:59:31.702073 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:59:31.702073 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:59:31.702073 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:59:31.702073 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:59:31.702073 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:59:31.702073 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:59:31.702073 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:59:31.702073 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:59:31.702073 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:59:31.702073 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 11:59:31.702073 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:59:31.702073 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:59:31.702073 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:59:31.702073 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:59:31.703513 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:59:31.703513 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:59:31.703513 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:59:31.703513 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:59:31.703513 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:59:31.703513 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:59:31.703513 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:59:31.703513 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:59:31.703513 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:59:31.703513 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:59:31.703513 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 11:59:31.703513 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 11:59:31.703513 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 175/343] 10.222.0.49 () {56 vars in 1114 bytes} [Fri Sep 9 11:59:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:59:31.704157 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0727be5f-03ec-421a-a43a-2a8322c87cf6 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:31.704221 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf351f65-a0aa-4fd0-8644-2672c04e97d8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:31.705408 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf351f65-a0aa-4fd0-8644-2672c04e97d8 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:31.889391 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 169/344] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 9 11:59:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 536 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 11:59:31.900545 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf351f65-a0aa-4fd0-8644-2672c04e97d8 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:31.919281 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:31.928597 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0727be5f-03ec-421a-a43a-2a8322c87cf6 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:32.000055 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2c0414-9303-4727-bb4c-2a5011cf64ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:32.000706 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2c0414-9303-4727-bb4c-2a5011cf64ab None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:32.000986 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2c0414-9303-4727-bb4c-2a5011cf64ab None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:32.001262 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2c0414-9303-4727-bb4c-2a5011cf64ab None admin] PATH_INFO: `/v3/roles/member` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:32.019891 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2c0414-9303-4727-bb4c-2a5011cf64ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:59:32.031559 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-be2c0414-9303-4727-bb4c-2a5011cf64ab None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 11:59:32.031559 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:59:32.031559 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:59:32.031559 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:59:32.031559 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:59:32.031559 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:59:32.031559 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:59:32.031559 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:59:32.031559 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:59:32.031559 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:59:32.031559 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:59:32.031559 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:59:32.031559 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:59:32.031559 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:59:32.031559 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:59:32.031559 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:59:32.032500 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:59:32.032500 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:59:32.032500 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:59:32.032500 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:59:32.032500 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:59:32.032500 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:59:32.032500 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:59:32.032500 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:59:32.032500 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:59:32.032500 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:59:32.032500 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:59:32.032500 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 11:59:32.032500 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:59:32.032500 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:59:32.032500 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:59:32.032500 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:59:32.033289 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:59:32.033289 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:59:32.033289 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:59:32.033289 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:59:32.033289 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:59:32.033289 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:59:32.033289 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:59:32.033289 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:59:32.033289 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:59:32.033289 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:59:32.033289 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 11:59:32.033289 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 11:59:32.033289 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 176/345] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 9 11:59:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:59:32.033937 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2c0414-9303-4727-bb4c-2a5011cf64ab None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:32.054372 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:32.101083 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf351f65-a0aa-4fd0-8644-2672c04e97d8 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:32.174390 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8004c21-4ef0-4d22-ba69-445eddfa6143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:32.175055 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8004c21-4ef0-4d22-ba69-445eddfa6143 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:32.175337 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8004c21-4ef0-4d22-ba69-445eddfa6143 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:32.175607 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8004c21-4ef0-4d22-ba69-445eddfa6143 None admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:32.176284 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8004c21-4ef0-4d22-ba69-445eddfa6143 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:59:32.176584 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8004c21-4ef0-4d22-ba69-445eddfa6143 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:32.178550 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8004c21-4ef0-4d22-ba69-445eddfa6143 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:32.198860 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2c0414-9303-4727-bb4c-2a5011cf64ab None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:32.199343 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 170/346] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 9 11:59:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:32.200293 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8004c21-4ef0-4d22-ba69-445eddfa6143 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:32.261571 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a39288-a618-4095-997e-7bda9dc920c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:32.262202 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a39288-a618-4095-997e-7bda9dc920c9 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:32.262415 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a39288-a618-4095-997e-7bda9dc920c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:32.262684 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a39288-a618-4095-997e-7bda9dc920c9 None admin] PATH_INFO: `/v3/projects/f8cf6274da2e456eb20ee0ba0d187a83` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:32.264530 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a39288-a618-4095-997e-7bda9dc920c9 None admin] RBAC: Authorizing `identity:get_project(project_id=f8cf6274da2e456eb20ee0ba0d187a83)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:32.266002 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a39288-a618-4095-997e-7bda9dc920c9 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:32.268868 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 177/347] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Sep 9 11:59:32 2022] GET /identity/v3/projects/f8cf6274da2e456eb20ee0ba0d187a83 => generated 298 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:32.269645 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a39288-a618-4095-997e-7bda9dc920c9 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:32.324605 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:32.384431 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8004c21-4ef0-4d22-ba69-445eddfa6143 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:32.390111 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:32.421617 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a39288-a618-4095-997e-7bda9dc920c9 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:32.438750 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f34739-3563-455a-8ce6-148de58e8b41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:32.439698 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f34739-3563-455a-8ce6-148de58e8b41 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:32.439979 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f34739-3563-455a-8ce6-148de58e8b41 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:32.440253 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f34739-3563-455a-8ce6-148de58e8b41 None admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:32.440933 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f34739-3563-455a-8ce6-148de58e8b41 None admin] RBAC: Adding query filter params (name=member) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:59:32.441237 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f34739-3563-455a-8ce6-148de58e8b41 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:32.442670 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f34739-3563-455a-8ce6-148de58e8b41 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:32.461972 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 171/348] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Sep 9 11:59:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:32.462813 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f34739-3563-455a-8ce6-148de58e8b41 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:32.470317 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddde307-714f-439d-b2d3-e42fd3492b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:32.471205 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddde307-714f-439d-b2d3-e42fd3492b56 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:32.472378 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddde307-714f-439d-b2d3-e42fd3492b56 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:32.472672 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddde307-714f-439d-b2d3-e42fd3492b56 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:32.566386 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddde307-714f-439d-b2d3-e42fd3492b56 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:32.568103 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddde307-714f-439d-b2d3-e42fd3492b56 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:32.604202 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f34739-3563-455a-8ce6-148de58e8b41 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:32.636119 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:32.656230 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dfb2d5-c294-44a2-8fd5-650c7e1cf5a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:32.656875 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dfb2d5-c294-44a2-8fd5-650c7e1cf5a2 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:32.657155 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dfb2d5-c294-44a2-8fd5-650c7e1cf5a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:32.657427 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dfb2d5-c294-44a2-8fd5-650c7e1cf5a2 None admin] PATH_INFO: `/v3/groups/09faa8b4bc694f2aa6c4a00e0736338e` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:32.664941 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dfb2d5-c294-44a2-8fd5-650c7e1cf5a2 None admin] RBAC: Authorizing `identity:get_group(group_id=09faa8b4bc694f2aa6c4a00e0736338e)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:32.666424 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dfb2d5-c294-44a2-8fd5-650c7e1cf5a2 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:32.668862 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 172/349] 10.222.0.49 () {56 vars in 1180 bytes} [Fri Sep 9 11:59:32 2022] GET /identity/v3/groups/09faa8b4bc694f2aa6c4a00e0736338e => generated 226 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:32.669592 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ef1b73-2b1c-4fb4-90fa-701629775380 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:32.669875 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ef1b73-2b1c-4fb4-90fa-701629775380 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:32.671266 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ef1b73-2b1c-4fb4-90fa-701629775380 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:32.676362 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 173/350] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 11:59:32 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:59:32.677422 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:32.677531 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ef1b73-2b1c-4fb4-90fa-701629775380 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:32.718083 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 178/351] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 9 11:59:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 11:59:32.719649 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b19036c-8f73-47c5-b554-b25cad6361fe None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:32.720016 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b19036c-8f73-47c5-b554-b25cad6361fe None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:32.720212 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b19036c-8f73-47c5-b554-b25cad6361fe None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:32.721830 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 179/352] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 11:59:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:59:32.722513 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b19036c-8f73-47c5-b554-b25cad6361fe None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:32.836256 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ef1b73-2b1c-4fb4-90fa-701629775380 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:32.837882 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b19036c-8f73-47c5-b554-b25cad6361fe None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:32.875403 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb6402e-a100-40b2-8e4a-ed9c118455f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:32.876012 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb6402e-a100-40b2-8e4a-ed9c118455f6 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:32.876290 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb6402e-a100-40b2-8e4a-ed9c118455f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:32.876554 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb6402e-a100-40b2-8e4a-ed9c118455f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:32.877276 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb6402e-a100-40b2-8e4a-ed9c118455f6 None admin] RBAC: Adding query filter params (group.id=09faa8b4bc694f2aa6c4a00e0736338e, role.id=eecd2f160dd844f5906f3d345780c501, scope.project.id=f8cf6274da2e456eb20ee0ba0d187a83) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:59:32.877574 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb6402e-a100-40b2-8e4a-ed9c118455f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:32.879038 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb6402e-a100-40b2-8e4a-ed9c118455f6 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:32.881711 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfb9b2e-fd88-48dc-bfdb-8d4d3c232381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:32.882320 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfb9b2e-fd88-48dc-bfdb-8d4d3c232381 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:32.882593 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfb9b2e-fd88-48dc-bfdb-8d4d3c232381 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:32.882894 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfb9b2e-fd88-48dc-bfdb-8d4d3c232381 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:32.887462 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 180/353] 10.222.0.49 () {56 vars in 1376 bytes} [Fri Sep 9 11:59:32 2022] GET /identity/v3/role_assignments?group.id=09faa8b4bc694f2aa6c4a00e0736338e&scope.project.id=f8cf6274da2e456eb20ee0ba0d187a83&role.id=eecd2f160dd844f5906f3d345780c501 => generated 260 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:32.889599 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453e1a5d-bfdf-4e7e-8a6f-b6d44891a329 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:32.889883 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453e1a5d-bfdf-4e7e-8a6f-b6d44891a329 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:32.890152 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453e1a5d-bfdf-4e7e-8a6f-b6d44891a329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:32.914533 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-453e1a5d-bfdf-4e7e-8a6f-b6d44891a329 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:59:32.915951 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-453e1a5d-bfdf-4e7e-8a6f-b6d44891a329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:32.963788 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:32.967920 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:32.985369 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfb9b2e-fd88-48dc-bfdb-8d4d3c232381 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:32.987018 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfb9b2e-fd88-48dc-bfdb-8d4d3c232381 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:33.009809 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 181/354] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 11:59:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:59:33.011533 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9b4b2f-643a-4acf-8b58-a897d32a52ab None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:33.012000 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9b4b2f-643a-4acf-8b58-a897d32a52ab None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:33.012445 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9b4b2f-643a-4acf-8b58-a897d32a52ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:33.035255 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-ca9b4b2f-643a-4acf-8b58-a897d32a52ab None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:59:33.037480 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-ca9b4b2f-643a-4acf-8b58-a897d32a52ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:33.121592 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 182/355] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 11:59:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:59:33.122356 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9b4b2f-643a-4acf-8b58-a897d32a52ab None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:33.134684 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 174/356] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 9 11:59:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 11:59:33.136402 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06a4b03-1b79-49f0-9703-ef0b33440a7e None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:33.136692 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06a4b03-1b79-49f0-9703-ef0b33440a7e None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:33.136965 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06a4b03-1b79-49f0-9703-ef0b33440a7e None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:33.138461 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 175/357] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 11:59:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:59:33.139952 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06a4b03-1b79-49f0-9703-ef0b33440a7e None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:33.261949 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:33.270261 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06a4b03-1b79-49f0-9703-ef0b33440a7e None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:33.311664 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34935f7-c132-4cd7-87cf-6f9bed127b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:33.312462 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34935f7-c132-4cd7-87cf-6f9bed127b7d None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:33.312739 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34935f7-c132-4cd7-87cf-6f9bed127b7d None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:33.313006 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34935f7-c132-4cd7-87cf-6f9bed127b7d None admin] PATH_INFO: `/v3/projects/5f8dc54c23f543d197444a43c6b82c01` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:33.315346 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34935f7-c132-4cd7-87cf-6f9bed127b7d None admin] RBAC: Authorizing `identity:get_project(project_id=5f8dc54c23f543d197444a43c6b82c01)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:33.317016 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34935f7-c132-4cd7-87cf-6f9bed127b7d None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:33.320222 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 176/358] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Sep 9 11:59:33 2022] GET /identity/v3/projects/5f8dc54c23f543d197444a43c6b82c01 => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:33.321453 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8af631-00ec-49aa-bcad-d6b70637ee56 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:33.321789 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8af631-00ec-49aa-bcad-d6b70637ee56 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:33.322192 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8af631-00ec-49aa-bcad-d6b70637ee56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:33.335678 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9b4b2f-643a-4acf-8b58-a897d32a52ab None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:33.337309 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:33.355966 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-6f8af631-00ec-49aa-bcad-d6b70637ee56 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:59:33.357219 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-6f8af631-00ec-49aa-bcad-d6b70637ee56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:33.374156 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24eac7ce-4aac-4e4f-a747-cb1ac4bb028e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:33.374731 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eac7ce-4aac-4e4f-a747-cb1ac4bb028e None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:33.375038 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eac7ce-4aac-4e4f-a747-cb1ac4bb028e None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:33.375313 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eac7ce-4aac-4e4f-a747-cb1ac4bb028e None admin] PATH_INFO: `/v3/projects/f8cf6274da2e456eb20ee0ba0d187a83` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:33.376433 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24eac7ce-4aac-4e4f-a747-cb1ac4bb028e None admin] RBAC: Authorizing `identity:get_project(project_id=f8cf6274da2e456eb20ee0ba0d187a83)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:33.377868 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24eac7ce-4aac-4e4f-a747-cb1ac4bb028e None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:33.379532 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 183/359] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Sep 9 11:59:33 2022] GET /identity/v3/projects/f8cf6274da2e456eb20ee0ba0d187a83 => generated 298 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:33.381065 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24eac7ce-4aac-4e4f-a747-cb1ac4bb028e None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:33.488773 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 177/360] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 11:59:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:59:33.490282 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8af631-00ec-49aa-bcad-d6b70637ee56 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:33.497667 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24eac7ce-4aac-4e4f-a747-cb1ac4bb028e None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:33.534474 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c94fe2d-8352-4667-9d96-fdf93a803253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:33.534680 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c94fe2d-8352-4667-9d96-fdf93a803253 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:33.534978 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c94fe2d-8352-4667-9d96-fdf93a803253 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:33.535212 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c94fe2d-8352-4667-9d96-fdf93a803253 None admin] PATH_INFO: `/v3/groups/09faa8b4bc694f2aa6c4a00e0736338e` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:33.536474 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c94fe2d-8352-4667-9d96-fdf93a803253 None admin] RBAC: Authorizing `identity:get_group(group_id=09faa8b4bc694f2aa6c4a00e0736338e)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:33.538235 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c94fe2d-8352-4667-9d96-fdf93a803253 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:33.540180 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 184/361] 10.222.0.49 () {56 vars in 1180 bytes} [Fri Sep 9 11:59:33 2022] GET /identity/v3/groups/09faa8b4bc694f2aa6c4a00e0736338e => generated 226 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:33.541520 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c94fe2d-8352-4667-9d96-fdf93a803253 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:33.600031 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:33.604601 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:33.613748 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8af631-00ec-49aa-bcad-d6b70637ee56 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:33.662108 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3879fe65-8f9f-494a-952b-2db5f3cad887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:33.662974 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3879fe65-8f9f-494a-952b-2db5f3cad887 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:33.663257 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3879fe65-8f9f-494a-952b-2db5f3cad887 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:33.663525 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3879fe65-8f9f-494a-952b-2db5f3cad887 None admin] PATH_INFO: `/v3/groups/09faa8b4bc694f2aa6c4a00e0736338e` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:33.664660 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3879fe65-8f9f-494a-952b-2db5f3cad887 None admin] RBAC: Authorizing `identity:get_group(group_id=09faa8b4bc694f2aa6c4a00e0736338e)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:33.666165 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3879fe65-8f9f-494a-952b-2db5f3cad887 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:33.668412 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 178/362] 10.222.0.49 () {56 vars in 1180 bytes} [Fri Sep 9 11:59:33 2022] GET /identity/v3/groups/09faa8b4bc694f2aa6c4a00e0736338e => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:33.668970 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c94fe2d-8352-4667-9d96-fdf93a803253 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:33.669256 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3879fe65-8f9f-494a-952b-2db5f3cad887 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:33.715481 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d31439-38c5-48f3-9a9d-89e59ac317d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:33.716122 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d31439-38c5-48f3-9a9d-89e59ac317d9 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:33.716970 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d31439-38c5-48f3-9a9d-89e59ac317d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:33.717248 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d31439-38c5-48f3-9a9d-89e59ac317d9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:33.730081 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d31439-38c5-48f3-9a9d-89e59ac317d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:59:33.736160 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-56d31439-38c5-48f3-9a9d-89e59ac317d9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 11:59:33.736160 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:59:33.736160 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:59:33.736160 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:59:33.736160 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:59:33.736160 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:59:33.736160 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:59:33.736160 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:59:33.736160 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:59:33.736160 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:59:33.736160 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:59:33.736160 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:59:33.736160 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:59:33.736160 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:59:33.736160 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:59:33.736160 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:59:33.737017 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:59:33.737017 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:59:33.737017 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:59:33.737017 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:59:33.737017 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:59:33.737017 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:59:33.737017 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:59:33.737017 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:59:33.737017 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:59:33.737017 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:59:33.737017 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:59:33.737017 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 11:59:33.737017 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:59:33.737017 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:59:33.737017 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:59:33.737017 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:59:33.737808 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:59:33.737808 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:59:33.737808 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:59:33.737808 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:59:33.737808 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:59:33.737808 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:59:33.737808 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:59:33.737808 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:59:33.737808 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:59:33.737808 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:59:33.737808 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 11:59:33.737808 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 11:59:33.737808 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 185/363] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 9 11:59:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:59:33.738447 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d31439-38c5-48f3-9a9d-89e59ac317d9 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:33.796427 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3879fe65-8f9f-494a-952b-2db5f3cad887 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:33.839341 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a468015-9f8e-49ef-8def-4604850beca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:33.840078 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a468015-9f8e-49ef-8def-4604850beca1 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:33.840355 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a468015-9f8e-49ef-8def-4604850beca1 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:33.840627 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a468015-9f8e-49ef-8def-4604850beca1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:33.852766 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:33.856911 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d31439-38c5-48f3-9a9d-89e59ac317d9 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:33.858082 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a468015-9f8e-49ef-8def-4604850beca1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:59:33.865490 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-2a468015-9f8e-49ef-8def-4604850beca1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 11:59:33.865490 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:59:33.865490 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:59:33.865490 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:59:33.865490 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:59:33.865490 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:59:33.865490 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:59:33.865490 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:59:33.865490 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:59:33.865490 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:59:33.865490 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:59:33.865490 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:59:33.865490 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:59:33.865490 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:59:33.865490 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:59:33.865490 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:59:33.866421 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:59:33.866421 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:59:33.866421 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:59:33.866421 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:59:33.866421 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:59:33.866421 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:59:33.866421 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:59:33.866421 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:59:33.866421 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:59:33.866421 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:59:33.866421 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:59:33.866421 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 11:59:33.866421 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:59:33.866421 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:59:33.866421 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:59:33.866421 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:59:33.867739 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:59:33.867739 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:59:33.867739 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:59:33.867739 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:59:33.867739 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:59:33.867739 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:59:33.867739 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:59:33.867739 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:59:33.867739 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:59:33.867739 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:59:33.867739 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 11:59:33.867739 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 11:59:33.867739 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 179/364] 10.222.0.49 () {56 vars in 1114 bytes} [Fri Sep 9 11:59:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:59:33.868580 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a468015-9f8e-49ef-8def-4604850beca1 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:33.889589 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:33.903891 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2234ce6-053a-4b54-8c10-1af34356219c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:33.903891 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2234ce6-053a-4b54-8c10-1af34356219c None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:33.904123 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2234ce6-053a-4b54-8c10-1af34356219c None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:33.904256 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2234ce6-053a-4b54-8c10-1af34356219c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:33.905082 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2234ce6-053a-4b54-8c10-1af34356219c None admin] RBAC: Adding query filter params (group.id=09faa8b4bc694f2aa6c4a00e0736338e, role.id=eecd2f160dd844f5906f3d345780c501, scope.project.id=5f8dc54c23f543d197444a43c6b82c01) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:59:33.905480 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2234ce6-053a-4b54-8c10-1af34356219c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:33.906982 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2234ce6-053a-4b54-8c10-1af34356219c None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:33.913033 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 186/365] 10.222.0.49 () {56 vars in 1376 bytes} [Fri Sep 9 11:59:33 2022] GET /identity/v3/role_assignments?group.id=09faa8b4bc694f2aa6c4a00e0736338e&scope.project.id=5f8dc54c23f543d197444a43c6b82c01&role.id=eecd2f160dd844f5906f3d345780c501 => generated 260 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:33.913841 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2234ce6-053a-4b54-8c10-1af34356219c None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:33.995701 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a468015-9f8e-49ef-8def-4604850beca1 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:34.034463 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db32a0a-2d93-4600-b263-aff945fb2e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:34.034809 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2234ce6-053a-4b54-8c10-1af34356219c None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:34.035682 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db32a0a-2d93-4600-b263-aff945fb2e0a None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:34.036708 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db32a0a-2d93-4600-b263-aff945fb2e0a None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:34.037402 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db32a0a-2d93-4600-b263-aff945fb2e0a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:34.038419 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db32a0a-2d93-4600-b263-aff945fb2e0a None admin] RBAC: Adding query filter params (group.id=09faa8b4bc694f2aa6c4a00e0736338e, role.id=a7c71d2d9914408bb9953f85329d1e80, scope.project.id=f8cf6274da2e456eb20ee0ba0d187a83) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:59:34.038959 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db32a0a-2d93-4600-b263-aff945fb2e0a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:34.040613 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db32a0a-2d93-4600-b263-aff945fb2e0a None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:34.046948 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 180/366] 10.222.0.49 () {56 vars in 1376 bytes} [Fri Sep 9 11:59:33 2022] GET /identity/v3/role_assignments?group.id=09faa8b4bc694f2aa6c4a00e0736338e&scope.project.id=f8cf6274da2e456eb20ee0ba0d187a83&role.id=a7c71d2d9914408bb9953f85329d1e80 => generated 260 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:34.048099 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db32a0a-2d93-4600-b263-aff945fb2e0a None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:34.096503 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102f97b2-8168-4ae8-a21b-e09aae562667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:34.096964 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102f97b2-8168-4ae8-a21b-e09aae562667 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:34.097244 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102f97b2-8168-4ae8-a21b-e09aae562667 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:34.097517 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102f97b2-8168-4ae8-a21b-e09aae562667 None admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:34.098201 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102f97b2-8168-4ae8-a21b-e09aae562667 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:59:34.098503 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102f97b2-8168-4ae8-a21b-e09aae562667 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:34.100567 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102f97b2-8168-4ae8-a21b-e09aae562667 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:34.122736 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 187/367] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Sep 9 11:59:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:34.123497 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102f97b2-8168-4ae8-a21b-e09aae562667 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:34.148613 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:34.172335 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db32a0a-2d93-4600-b263-aff945fb2e0a None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:34.173263 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:34.215134 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f44179-5d33-4cff-9184-770e0c508fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:34.215805 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f44179-5d33-4cff-9184-770e0c508fb8 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:34.216080 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f44179-5d33-4cff-9184-770e0c508fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:34.216350 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f44179-5d33-4cff-9184-770e0c508fb8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:34.229200 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f44179-5d33-4cff-9184-770e0c508fb8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:59:34.235371 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-41f44179-5d33-4cff-9184-770e0c508fb8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:59:34.235371 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:59:34.235371 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:59:34.235371 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:59:34.235371 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:59:34.235371 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:59:34.235371 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:59:34.235371 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:59:34.235371 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:59:34.235371 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:59:34.235371 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:59:34.235371 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:59:34.235371 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:59:34.235371 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:59:34.235371 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:59:34.235371 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:59:34.236401 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:59:34.236401 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:59:34.236401 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:59:34.236401 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:59:34.236401 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:59:34.236401 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:59:34.236401 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:59:34.236401 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:59:34.236401 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:59:34.236401 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:59:34.236401 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:59:34.236401 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 11:59:34.236401 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:59:34.236401 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:59:34.236401 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:59:34.236401 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:59:34.237189 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:59:34.237189 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:59:34.237189 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:59:34.237189 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:59:34.237189 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:59:34.237189 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:59:34.237189 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:59:34.237189 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:59:34.237189 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:59:34.237189 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:59:34.237189 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:59:34.237189 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 11:59:34.237189 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 181/368] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Sep 9 11:59:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:59:34.237825 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f44179-5d33-4cff-9184-770e0c508fb8 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:34.324183 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102f97b2-8168-4ae8-a21b-e09aae562667 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:34.354103 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f44179-5d33-4cff-9184-770e0c508fb8 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:34.387715 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5680992c-e8c2-4316-9891-2f845b80c760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:34.388346 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5680992c-e8c2-4316-9891-2f845b80c760 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:34.388625 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5680992c-e8c2-4316-9891-2f845b80c760 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:34.388899 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5680992c-e8c2-4316-9891-2f845b80c760 None admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:34.389582 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5680992c-e8c2-4316-9891-2f845b80c760 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:59:34.389888 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5680992c-e8c2-4316-9891-2f845b80c760 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:34.391286 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5680992c-e8c2-4316-9891-2f845b80c760 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:34.392553 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0185fd-6d6e-4548-92cb-95038c86aaa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:34.393347 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0185fd-6d6e-4548-92cb-95038c86aaa7 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:34.393763 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0185fd-6d6e-4548-92cb-95038c86aaa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:34.394192 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0185fd-6d6e-4548-92cb-95038c86aaa7 None admin] PATH_INFO: `/v3/users/3527c7a46a3148b882c3511c9febe789` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:34.396117 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0185fd-6d6e-4548-92cb-95038c86aaa7 None admin] RBAC: Authorizing `identity:get_user(user_id=3527c7a46a3148b882c3511c9febe789)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:34.398090 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0185fd-6d6e-4548-92cb-95038c86aaa7 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:34.400209 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 182/369] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Sep 9 11:59:34 2022] GET /identity/v3/users/3527c7a46a3148b882c3511c9febe789 => generated 293 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:34.400884 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 188/370] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 9 11:59:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:34.401357 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0185fd-6d6e-4548-92cb-95038c86aaa7 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:34.406087 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5680992c-e8c2-4316-9891-2f845b80c760 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:34.414297 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:34.550489 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0185fd-6d6e-4548-92cb-95038c86aaa7 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:34.567273 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:34.591967 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5680992c-e8c2-4316-9891-2f845b80c760 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:34.624667 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8abbfc0-9c70-415d-a37f-c8fea9d2fd22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:34.625526 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8abbfc0-9c70-415d-a37f-c8fea9d2fd22 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:34.625939 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8abbfc0-9c70-415d-a37f-c8fea9d2fd22 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:34.626342 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8abbfc0-9c70-415d-a37f-c8fea9d2fd22 None admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:34.627161 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8abbfc0-9c70-415d-a37f-c8fea9d2fd22 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:59:34.627797 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8abbfc0-9c70-415d-a37f-c8fea9d2fd22 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:34.630816 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8abbfc0-9c70-415d-a37f-c8fea9d2fd22 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:34.647378 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 183/371] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 9 11:59:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:34.647378 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8abbfc0-9c70-415d-a37f-c8fea9d2fd22 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:34.654038 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c46438-7885-477d-bb3d-44784c762ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:34.654648 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c46438-7885-477d-bb3d-44784c762ca8 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:34.654932 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c46438-7885-477d-bb3d-44784c762ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:34.655197 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c46438-7885-477d-bb3d-44784c762ca8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:34.655921 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c46438-7885-477d-bb3d-44784c762ca8 None admin] RBAC: Adding query filter params (role.id=b1e7a1e6ddc54da7ba995fb55fd93016, user.id=3527c7a46a3148b882c3511c9febe789, scope.system=all) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:59:34.656219 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c46438-7885-477d-bb3d-44784c762ca8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:34.657648 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c46438-7885-477d-bb3d-44784c762ca8 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:34.678974 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 189/372] 10.222.0.49 () {56 vars in 1308 bytes} [Fri Sep 9 11:59:34 2022] GET /identity/v3/role_assignments?user.id=3527c7a46a3148b882c3511c9febe789&scope.system=all&role.id=b1e7a1e6ddc54da7ba995fb55fd93016 => generated 510 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:59:34.679942 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c46438-7885-477d-bb3d-44784c762ca8 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:34.745876 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:34.782953 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8abbfc0-9c70-415d-a37f-c8fea9d2fd22 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:34.792259 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c46438-7885-477d-bb3d-44784c762ca8 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:34.836085 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357a44e9-2a92-4952-ac2e-ee7f4f03e733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:34.836655 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357a44e9-2a92-4952-ac2e-ee7f4f03e733 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:34.836939 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357a44e9-2a92-4952-ac2e-ee7f4f03e733 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:34.837214 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357a44e9-2a92-4952-ac2e-ee7f4f03e733 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:34.839598 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdce497f-5302-4025-9a6e-8ab862783757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:34.840220 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdce497f-5302-4025-9a6e-8ab862783757 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:34.840497 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdce497f-5302-4025-9a6e-8ab862783757 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:34.840768 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdce497f-5302-4025-9a6e-8ab862783757 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:34.879320 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:34.987162 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdce497f-5302-4025-9a6e-8ab862783757 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:34.988616 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdce497f-5302-4025-9a6e-8ab862783757 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:34.988726 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357a44e9-2a92-4952-ac2e-ee7f4f03e733 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:34.990475 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357a44e9-2a92-4952-ac2e-ee7f4f03e733 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:35.109833 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:35.158081 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 190/373] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 9 11:59:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 478 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 11:59:35.159618 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 184/374] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 9 11:59:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 513 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 11:59:35.169126 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdce497f-5302-4025-9a6e-8ab862783757 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:35.169552 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357a44e9-2a92-4952-ac2e-ee7f4f03e733 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:35.234138 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:35.303082 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357a44e9-2a92-4952-ac2e-ee7f4f03e733 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:35.327860 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdce497f-5302-4025-9a6e-8ab862783757 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:35.351060 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd56c2d-c1ec-4cef-90b5-db9f98d3d71a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:35.352250 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd56c2d-c1ec-4cef-90b5-db9f98d3d71a None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:35.352536 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd56c2d-c1ec-4cef-90b5-db9f98d3d71a None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:35.352814 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd56c2d-c1ec-4cef-90b5-db9f98d3d71a None admin] PATH_INFO: `/v3/projects/ae50911c7cf74f75b5b41fa5275c8468` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:35.354158 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd56c2d-c1ec-4cef-90b5-db9f98d3d71a None admin] RBAC: Authorizing `identity:get_project(project_id=ae50911c7cf74f75b5b41fa5275c8468)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:35.355835 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd56c2d-c1ec-4cef-90b5-db9f98d3d71a None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:35.358537 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 185/375] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Sep 9 11:59:35 2022] GET /identity/v3/projects/ae50911c7cf74f75b5b41fa5275c8468 => generated 344 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:59:35.364268 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd56c2d-c1ec-4cef-90b5-db9f98d3d71a None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:35.413512 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdfd849-0875-4628-ada2-43b8b0708d9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:35.414269 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdfd849-0875-4628-ada2-43b8b0708d9f None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:35.414433 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdfd849-0875-4628-ada2-43b8b0708d9f None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:35.414679 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdfd849-0875-4628-ada2-43b8b0708d9f None admin] PATH_INFO: `/v3/projects/5f8dc54c23f543d197444a43c6b82c01` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:35.415872 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdfd849-0875-4628-ada2-43b8b0708d9f None admin] RBAC: Authorizing `identity:get_project(project_id=5f8dc54c23f543d197444a43c6b82c01)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:35.417332 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdfd849-0875-4628-ada2-43b8b0708d9f None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:35.418072 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:35.419446 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 191/376] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Sep 9 11:59:35 2022] GET /identity/v3/projects/5f8dc54c23f543d197444a43c6b82c01 => generated 302 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:35.428517 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdfd849-0875-4628-ada2-43b8b0708d9f None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:35.494854 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd56c2d-c1ec-4cef-90b5-db9f98d3d71a None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:35.541699 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcbb15f-9e7e-4d57-8247-01f52f9d05c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:35.542725 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcbb15f-9e7e-4d57-8247-01f52f9d05c0 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:35.543207 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcbb15f-9e7e-4d57-8247-01f52f9d05c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:35.543827 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcbb15f-9e7e-4d57-8247-01f52f9d05c0 None admin] PATH_INFO: `/v3/groups/1f810b1ba4b7474589f25485737977bb` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:35.549474 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcbb15f-9e7e-4d57-8247-01f52f9d05c0 None admin] RBAC: Authorizing `identity:get_group(group_id=1f810b1ba4b7474589f25485737977bb)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:35.551156 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcbb15f-9e7e-4d57-8247-01f52f9d05c0 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:35.554382 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 186/377] 10.222.0.49 () {56 vars in 1180 bytes} [Fri Sep 9 11:59:35 2022] GET /identity/v3/groups/1f810b1ba4b7474589f25485737977bb => generated 229 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:59:35.560344 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcbb15f-9e7e-4d57-8247-01f52f9d05c0 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:35.657153 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdfd849-0875-4628-ada2-43b8b0708d9f None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:35.682549 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:35.720947 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4926868f-80f9-4160-912d-9255785a661c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:35.721585 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4926868f-80f9-4160-912d-9255785a661c None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:35.721867 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4926868f-80f9-4160-912d-9255785a661c None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:35.722141 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4926868f-80f9-4160-912d-9255785a661c None admin] PATH_INFO: `/v3/groups/09faa8b4bc694f2aa6c4a00e0736338e` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:35.724397 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4926868f-80f9-4160-912d-9255785a661c None admin] RBAC: Authorizing `identity:get_group(group_id=09faa8b4bc694f2aa6c4a00e0736338e)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:35.725871 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4926868f-80f9-4160-912d-9255785a661c None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:35.727512 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 192/378] 10.222.0.49 () {56 vars in 1180 bytes} [Fri Sep 9 11:59:35 2022] GET /identity/v3/groups/09faa8b4bc694f2aa6c4a00e0736338e => generated 226 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:59:35.729621 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcbb15f-9e7e-4d57-8247-01f52f9d05c0 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:35.732902 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:35.733743 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4926868f-80f9-4160-912d-9255785a661c None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:35.798687 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fd5928-d318-4024-8d3e-bf9a0c084228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:35.799339 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fd5928-d318-4024-8d3e-bf9a0c084228 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:35.799618 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fd5928-d318-4024-8d3e-bf9a0c084228 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:35.799885 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fd5928-d318-4024-8d3e-bf9a0c084228 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:35.800609 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fd5928-d318-4024-8d3e-bf9a0c084228 None admin] RBAC: Adding query filter params (group.id=1f810b1ba4b7474589f25485737977bb, role.id=3f2ca33902a04b6dbc2ef7d06d7289d3, scope.project.id=ae50911c7cf74f75b5b41fa5275c8468) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:59:35.800912 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fd5928-d318-4024-8d3e-bf9a0c084228 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:35.802885 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fd5928-d318-4024-8d3e-bf9a0c084228 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:35.815999 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 187/379] 10.222.0.49 () {56 vars in 1376 bytes} [Fri Sep 9 11:59:35 2022] GET /identity/v3/role_assignments?group.id=1f810b1ba4b7474589f25485737977bb&scope.project.id=ae50911c7cf74f75b5b41fa5275c8468&role.id=3f2ca33902a04b6dbc2ef7d06d7289d3 => generated 260 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:59:35.888364 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4926868f-80f9-4160-912d-9255785a661c None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:35.929412 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db9073f-7967-442c-ae6f-90e2a517a6fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:35.929949 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db9073f-7967-442c-ae6f-90e2a517a6fa None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:35.930230 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db9073f-7967-442c-ae6f-90e2a517a6fa None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:35.930505 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db9073f-7967-442c-ae6f-90e2a517a6fa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:35.931276 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db9073f-7967-442c-ae6f-90e2a517a6fa None admin] RBAC: Adding query filter params (group.id=09faa8b4bc694f2aa6c4a00e0736338e, role.id=a7c71d2d9914408bb9953f85329d1e80, scope.project.id=5f8dc54c23f543d197444a43c6b82c01) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:59:35.931582 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db9073f-7967-442c-ae6f-90e2a517a6fa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:35.933044 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db9073f-7967-442c-ae6f-90e2a517a6fa None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:35.939948 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 193/380] 10.222.0.49 () {56 vars in 1376 bytes} [Fri Sep 9 11:59:35 2022] GET /identity/v3/role_assignments?group.id=09faa8b4bc694f2aa6c4a00e0736338e&scope.project.id=5f8dc54c23f543d197444a43c6b82c01&role.id=a7c71d2d9914408bb9953f85329d1e80 => generated 260 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:35.984775 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d21b0d-d567-406b-853e-f5c9f5b2d2a9 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:35.985091 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d21b0d-d567-406b-853e-f5c9f5b2d2a9 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:35.985333 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d21b0d-d567-406b-853e-f5c9f5b2d2a9 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:35.986848 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 188/381] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 11:59:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:59:35.993780 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075b17dc-dff0-4e9e-bf3f-f5f89ca69b03 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:35.994093 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075b17dc-dff0-4e9e-bf3f-f5f89ca69b03 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:35.994330 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075b17dc-dff0-4e9e-bf3f-f5f89ca69b03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:36.023760 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-075b17dc-dff0-4e9e-bf3f-f5f89ca69b03 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:59:36.025132 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-075b17dc-dff0-4e9e-bf3f-f5f89ca69b03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:36.077653 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:36.110352 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 194/382] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 11:59:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:59:36.431808 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d21b0d-d567-406b-853e-f5c9f5b2d2a9 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:36.574793 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d21b0d-d567-406b-853e-f5c9f5b2d2a9 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:36.617758 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6574ba08-4f91-4398-ac2a-6417d7c4bc18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:36.618397 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6574ba08-4f91-4398-ac2a-6417d7c4bc18 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:36.618675 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6574ba08-4f91-4398-ac2a-6417d7c4bc18 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:36.618963 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6574ba08-4f91-4398-ac2a-6417d7c4bc18 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:36.634017 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6574ba08-4f91-4398-ac2a-6417d7c4bc18 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:59:36.639526 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-6574ba08-4f91-4398-ac2a-6417d7c4bc18 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 11:59:36.639526 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:59:36.639526 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:59:36.639526 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:59:36.639526 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:59:36.639526 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:59:36.639526 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:59:36.639526 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:59:36.639526 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:59:36.639526 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:59:36.639526 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:59:36.639526 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:59:36.639526 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:59:36.639526 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:59:36.639526 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:59:36.639526 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:59:36.640453 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:59:36.640453 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:59:36.640453 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:59:36.640453 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:59:36.640453 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:59:36.640453 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:59:36.640453 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:59:36.640453 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:59:36.640453 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:59:36.640453 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:59:36.640453 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:59:36.640453 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 11:59:36.640453 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:59:36.640453 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:59:36.640453 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:59:36.640453 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:59:36.641280 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:59:36.641280 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:59:36.641280 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:59:36.641280 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:59:36.641280 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:59:36.641280 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:59:36.641280 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:59:36.641280 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:59:36.641280 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:59:36.641280 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:59:36.641280 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 11:59:36.641280 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 11:59:36.641280 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 189/383] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 9 11:59:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:59:36.647607 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075b17dc-dff0-4e9e-bf3f-f5f89ca69b03 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:36.773031 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075b17dc-dff0-4e9e-bf3f-f5f89ca69b03 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:36.810778 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01da47c1-77bc-4278-b235-d03360df1936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:36.811445 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01da47c1-77bc-4278-b235-d03360df1936 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:36.811727 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01da47c1-77bc-4278-b235-d03360df1936 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:36.812000 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01da47c1-77bc-4278-b235-d03360df1936 None admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:36.812669 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01da47c1-77bc-4278-b235-d03360df1936 None admin] RBAC: Adding query filter params (name=member) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:59:36.812972 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01da47c1-77bc-4278-b235-d03360df1936 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:36.814349 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01da47c1-77bc-4278-b235-d03360df1936 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:36.826187 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 195/384] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Sep 9 11:59:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:36.831431 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6574ba08-4f91-4398-ac2a-6417d7c4bc18 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:36.886784 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:36.997309 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6574ba08-4f91-4398-ac2a-6417d7c4bc18 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:37.038557 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b08d4c4-c194-4ae0-8f1f-dcc482f82923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:37.039144 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b08d4c4-c194-4ae0-8f1f-dcc482f82923 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:37.039427 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b08d4c4-c194-4ae0-8f1f-dcc482f82923 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:37.039706 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b08d4c4-c194-4ae0-8f1f-dcc482f82923 None admin] PATH_INFO: `/v3/groups/09faa8b4bc694f2aa6c4a00e0736338e` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:37.043758 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b08d4c4-c194-4ae0-8f1f-dcc482f82923 None admin] RBAC: Authorizing `identity:get_group(group_id=09faa8b4bc694f2aa6c4a00e0736338e)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:37.045256 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b08d4c4-c194-4ae0-8f1f-dcc482f82923 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:37.047314 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 190/385] 10.222.0.49 () {56 vars in 1180 bytes} [Fri Sep 9 11:59:36 2022] GET /identity/v3/groups/09faa8b4bc694f2aa6c4a00e0736338e => generated 226 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:37.051724 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8489940-98af-4bb7-8b9c-7f8f55b48210 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:37.052260 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8489940-98af-4bb7-8b9c-7f8f55b48210 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:37.052780 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b08d4c4-c194-4ae0-8f1f-dcc482f82923 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:37.053039 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8489940-98af-4bb7-8b9c-7f8f55b48210 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:37.055104 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 196/386] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 11:59:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:59:37.061665 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4600c6-9541-4a1e-842e-4df8f5b2e0e4 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:37.062230 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4600c6-9541-4a1e-842e-4df8f5b2e0e4 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:37.062701 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4600c6-9541-4a1e-842e-4df8f5b2e0e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:37.089713 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-ef4600c6-9541-4a1e-842e-4df8f5b2e0e4 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:59:37.095452 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-ef4600c6-9541-4a1e-842e-4df8f5b2e0e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:37.137492 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:37.163170 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:37.172190 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b08d4c4-c194-4ae0-8f1f-dcc482f82923 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:37.196962 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 197/387] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 11:59:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:59:37.198584 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f96448c-d9f2-47d7-b283-981a368c3dbb None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:37.199110 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f96448c-d9f2-47d7-b283-981a368c3dbb None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:37.199598 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f96448c-d9f2-47d7-b283-981a368c3dbb None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:37.201067 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 198/388] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 11:59:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:59:37.206292 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bb2198-d72c-4294-ac63-1f2866e53bce None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:37.206606 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bb2198-d72c-4294-ac63-1f2866e53bce None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:37.206888 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bb2198-d72c-4294-ac63-1f2866e53bce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:37.220685 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e23112-ace5-469e-819d-1f0a7d1cefcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:37.221392 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e23112-ace5-469e-819d-1f0a7d1cefcd None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:37.221671 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e23112-ace5-469e-819d-1f0a7d1cefcd None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:37.221937 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e23112-ace5-469e-819d-1f0a7d1cefcd None admin] PATH_INFO: `/v3/projects/f8cf6274da2e456eb20ee0ba0d187a83` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:37.223114 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e23112-ace5-469e-819d-1f0a7d1cefcd None admin] RBAC: Authorizing `identity:get_project(project_id=f8cf6274da2e456eb20ee0ba0d187a83)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:37.225105 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e23112-ace5-469e-819d-1f0a7d1cefcd None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:37.227330 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 191/389] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Sep 9 11:59:37 2022] GET /identity/v3/projects/f8cf6274da2e456eb20ee0ba0d187a83 => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:59:37.240203 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e23112-ace5-469e-819d-1f0a7d1cefcd None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:37.243090 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-d5bb2198-d72c-4294-ac63-1f2866e53bce None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:59:37.244324 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-d5bb2198-d72c-4294-ac63-1f2866e53bce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:37.389399 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 199/390] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 11:59:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 11:59:37.395063 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e23112-ace5-469e-819d-1f0a7d1cefcd None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:37.432346 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e40cac8-a05b-49aa-a4f5-e618f3184536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:37.433054 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e40cac8-a05b-49aa-a4f5-e618f3184536 None admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:37.433328 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e40cac8-a05b-49aa-a4f5-e618f3184536 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:37.433598 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e40cac8-a05b-49aa-a4f5-e618f3184536 None admin] PATH_INFO: `/v3/projects/f8cf6274da2e456eb20ee0ba0d187a83/groups/09faa8b4bc694f2aa6c4a00e0736338e/roles/eecd2f160dd844f5906f3d345780c501` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:37.435503 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e40cac8-a05b-49aa-a4f5-e618f3184536 None admin] RBAC: Authorizing `identity:create_grant(project_id=f8cf6274da2e456eb20ee0ba0d187a83, group_id=09faa8b4bc694f2aa6c4a00e0736338e, role_id=eecd2f160dd844f5906f3d345780c501)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:37.437078 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e40cac8-a05b-49aa-a4f5-e618f3184536 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:37.453415 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 192/391] 10.222.0.49 () {58 vars in 1442 bytes} [Fri Sep 9 11:59:37 2022] PUT /identity/v3/projects/f8cf6274da2e456eb20ee0ba0d187a83/groups/09faa8b4bc694f2aa6c4a00e0736338e/roles/eecd2f160dd844f5906f3d345780c501 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:59:37.695069 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bb2198-d72c-4294-ac63-1f2866e53bce None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:37.766549 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e40cac8-a05b-49aa-a4f5-e618f3184536 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:37.799379 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:37.823571 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:37.831764 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bb2198-d72c-4294-ac63-1f2866e53bce None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:37.877314 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd3bfb7-7ca4-4682-85a8-94493889e0c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:37.878186 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd3bfb7-7ca4-4682-85a8-94493889e0c9 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:37.878706 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd3bfb7-7ca4-4682-85a8-94493889e0c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:37.879164 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd3bfb7-7ca4-4682-85a8-94493889e0c9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:37.902791 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e40cac8-a05b-49aa-a4f5-e618f3184536 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:37.904651 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd3bfb7-7ca4-4682-85a8-94493889e0c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:59:37.910678 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-dcd3bfb7-7ca4-4682-85a8-94493889e0c9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 11:59:37.910678 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:59:37.910678 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:59:37.910678 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:59:37.910678 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:59:37.910678 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:59:37.910678 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:59:37.910678 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:59:37.910678 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:59:37.910678 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:59:37.910678 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:59:37.910678 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:59:37.910678 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:59:37.910678 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:59:37.910678 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:59:37.910678 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:59:37.912060 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:59:37.912060 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:59:37.912060 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:59:37.912060 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:59:37.912060 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:59:37.912060 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:59:37.912060 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:59:37.912060 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:59:37.912060 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:59:37.912060 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:59:37.912060 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:59:37.912060 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 11:59:37.912060 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:59:37.912060 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:59:37.912060 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:59:37.912060 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:59:37.912860 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:59:37.912860 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:59:37.912860 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:59:37.912860 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:59:37.912860 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:59:37.912860 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:59:37.912860 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:59:37.912860 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:59:37.912860 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:59:37.912860 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:59:37.912860 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 11:59:37.912860 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 11:59:37.912860 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 200/392] 10.222.0.49 () {56 vars in 1114 bytes} [Fri Sep 9 11:59:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:59:37.928009 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd3bfb7-7ca4-4682-85a8-94493889e0c9 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:37.972426 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ce2326-6142-485c-9668-fd3a3f7ab824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:37.973067 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ce2326-6142-485c-9668-fd3a3f7ab824 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:37.973347 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ce2326-6142-485c-9668-fd3a3f7ab824 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:37.974177 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ce2326-6142-485c-9668-fd3a3f7ab824 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:37.996921 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ce2326-6142-485c-9668-fd3a3f7ab824 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:59:38.004923 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-69ce2326-6142-485c-9668-fd3a3f7ab824 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 11:59:38.004923 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:59:38.004923 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:59:38.004923 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:59:38.004923 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:59:38.004923 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:59:38.004923 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:59:38.004923 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:59:38.004923 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:59:38.004923 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:59:38.004923 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:59:38.004923 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:59:38.004923 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:59:38.004923 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:59:38.004923 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:59:38.004923 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:59:38.006314 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:59:38.006314 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:59:38.006314 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:59:38.006314 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:59:38.006314 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:59:38.006314 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:59:38.006314 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:59:38.006314 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:59:38.006314 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:59:38.006314 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:59:38.006314 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:59:38.006314 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 11:59:38.006314 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:59:38.006314 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:59:38.006314 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:59:38.006314 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:59:38.007108 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:59:38.007108 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:59:38.007108 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:59:38.007108 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:59:38.007108 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:59:38.007108 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:59:38.007108 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:59:38.007108 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:59:38.007108 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:59:38.007108 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:59:38.007108 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 11:59:38.007108 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 11:59:38.007108 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 193/393] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 9 11:59:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:59:38.015339 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ce2326-6142-485c-9668-fd3a3f7ab824 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:38.076455 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd3bfb7-7ca4-4682-85a8-94493889e0c9 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:38.138889 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:38.161538 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ce2326-6142-485c-9668-fd3a3f7ab824 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:38.168282 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1377f7d-cbbe-4878-93e9-8549b7cfd0a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:38.168911 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1377f7d-cbbe-4878-93e9-8549b7cfd0a8 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:38.169193 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1377f7d-cbbe-4878-93e9-8549b7cfd0a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:38.169463 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1377f7d-cbbe-4878-93e9-8549b7cfd0a8 None admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:38.170154 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1377f7d-cbbe-4878-93e9-8549b7cfd0a8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:59:38.170441 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1377f7d-cbbe-4878-93e9-8549b7cfd0a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:38.171860 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1377f7d-cbbe-4878-93e9-8549b7cfd0a8 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:38.193742 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 201/394] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 9 11:59:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:59:38.212011 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c7a2f8-6bfd-42cc-bc54-98d22128bd17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:38.212945 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c7a2f8-6bfd-42cc-bc54-98d22128bd17 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:38.213404 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c7a2f8-6bfd-42cc-bc54-98d22128bd17 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:38.213855 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c7a2f8-6bfd-42cc-bc54-98d22128bd17 None admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:38.214716 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c7a2f8-6bfd-42cc-bc54-98d22128bd17 None admin] RBAC: Adding query filter params (name=member) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:59:38.215216 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c7a2f8-6bfd-42cc-bc54-98d22128bd17 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:38.216767 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c7a2f8-6bfd-42cc-bc54-98d22128bd17 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:38.219901 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1377f7d-cbbe-4878-93e9-8549b7cfd0a8 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:38.244192 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 194/395] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Sep 9 11:59:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:38.248516 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:38.250271 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c7a2f8-6bfd-42cc-bc54-98d22128bd17 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:38.390716 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1377f7d-cbbe-4878-93e9-8549b7cfd0a8 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:38.429045 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301f87c7-26c2-4f51-9bb8-c82684b49f3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:38.430177 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301f87c7-26c2-4f51-9bb8-c82684b49f3f None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:38.430679 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301f87c7-26c2-4f51-9bb8-c82684b49f3f None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:38.431192 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301f87c7-26c2-4f51-9bb8-c82684b49f3f None admin] PATH_INFO: `/v3/groups/09faa8b4bc694f2aa6c4a00e0736338e` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:38.432559 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301f87c7-26c2-4f51-9bb8-c82684b49f3f None admin] RBAC: Authorizing `identity:get_group(group_id=09faa8b4bc694f2aa6c4a00e0736338e)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:38.434227 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301f87c7-26c2-4f51-9bb8-c82684b49f3f None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:38.436370 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 202/396] 10.222.0.49 () {56 vars in 1180 bytes} [Fri Sep 9 11:59:38 2022] GET /identity/v3/groups/09faa8b4bc694f2aa6c4a00e0736338e => generated 226 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:59:38.444457 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301f87c7-26c2-4f51-9bb8-c82684b49f3f None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:38.458951 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c7a2f8-6bfd-42cc-bc54-98d22128bd17 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:38.517079 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9df625-426b-47ba-91e3-87db20ac19ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:38.517716 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9df625-426b-47ba-91e3-87db20ac19ff None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:38.517992 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9df625-426b-47ba-91e3-87db20ac19ff None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:38.518267 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9df625-426b-47ba-91e3-87db20ac19ff None admin] PATH_INFO: `/v3/groups/09faa8b4bc694f2aa6c4a00e0736338e` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:38.519766 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9df625-426b-47ba-91e3-87db20ac19ff None admin] RBAC: Authorizing `identity:get_group(group_id=09faa8b4bc694f2aa6c4a00e0736338e)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:38.521757 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9df625-426b-47ba-91e3-87db20ac19ff None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:38.523886 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 195/397] 10.222.0.49 () {56 vars in 1180 bytes} [Fri Sep 9 11:59:38 2022] GET /identity/v3/groups/09faa8b4bc694f2aa6c4a00e0736338e => generated 226 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:38.528971 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9df625-426b-47ba-91e3-87db20ac19ff None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:38.542805 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:38.556163 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:38.561368 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301f87c7-26c2-4f51-9bb8-c82684b49f3f None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:38.601860 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847c9f77-bde5-48f2-8d0a-62a67aae257d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:38.602945 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847c9f77-bde5-48f2-8d0a-62a67aae257d None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:38.603239 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847c9f77-bde5-48f2-8d0a-62a67aae257d None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:38.603516 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847c9f77-bde5-48f2-8d0a-62a67aae257d None admin] PATH_INFO: `/v3/projects/f8cf6274da2e456eb20ee0ba0d187a83` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:38.605084 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847c9f77-bde5-48f2-8d0a-62a67aae257d None admin] RBAC: Authorizing `identity:get_project(project_id=f8cf6274da2e456eb20ee0ba0d187a83)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:38.606542 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847c9f77-bde5-48f2-8d0a-62a67aae257d None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:38.608519 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 203/398] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Sep 9 11:59:38 2022] GET /identity/v3/projects/f8cf6274da2e456eb20ee0ba0d187a83 => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:38.614182 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847c9f77-bde5-48f2-8d0a-62a67aae257d None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:38.734116 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847c9f77-bde5-48f2-8d0a-62a67aae257d None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:38.745006 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9df625-426b-47ba-91e3-87db20ac19ff None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:38.773234 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58abd3b-c7c2-4ab2-acbb-01b0ab2256fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:38.773934 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58abd3b-c7c2-4ab2-acbb-01b0ab2256fb None admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:38.774215 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58abd3b-c7c2-4ab2-acbb-01b0ab2256fb None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:38.774494 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58abd3b-c7c2-4ab2-acbb-01b0ab2256fb None admin] PATH_INFO: `/v3/projects/f8cf6274da2e456eb20ee0ba0d187a83/groups/09faa8b4bc694f2aa6c4a00e0736338e/roles/a7c71d2d9914408bb9953f85329d1e80` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:38.776769 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58abd3b-c7c2-4ab2-acbb-01b0ab2256fb None admin] RBAC: Authorizing `identity:create_grant(project_id=f8cf6274da2e456eb20ee0ba0d187a83, group_id=09faa8b4bc694f2aa6c4a00e0736338e, role_id=a7c71d2d9914408bb9953f85329d1e80)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:38.778211 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58abd3b-c7c2-4ab2-acbb-01b0ab2256fb None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:38.802865 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 204/399] 10.222.0.49 () {58 vars in 1442 bytes} [Fri Sep 9 11:59:38 2022] PUT /identity/v3/projects/f8cf6274da2e456eb20ee0ba0d187a83/groups/09faa8b4bc694f2aa6c4a00e0736338e/roles/a7c71d2d9914408bb9953f85329d1e80 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:59:38.811078 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134b9d85-2905-41db-a1d8-0719a00b3da9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:38.811879 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c68afa0-af58-4968-aa01-63423b2b15b0 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:38.812316 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c68afa0-af58-4968-aa01-63423b2b15b0 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:38.812707 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c68afa0-af58-4968-aa01-63423b2b15b0 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:38.812991 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134b9d85-2905-41db-a1d8-0719a00b3da9 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:38.813446 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134b9d85-2905-41db-a1d8-0719a00b3da9 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:38.813903 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134b9d85-2905-41db-a1d8-0719a00b3da9 None admin] PATH_INFO: `/v3/projects/5f8dc54c23f543d197444a43c6b82c01` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:38.814796 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 205/400] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 11:59:38 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:59:38.815962 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134b9d85-2905-41db-a1d8-0719a00b3da9 None admin] RBAC: Authorizing `identity:get_project(project_id=5f8dc54c23f543d197444a43c6b82c01)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:38.817827 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134b9d85-2905-41db-a1d8-0719a00b3da9 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:38.823215 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c509b92-47d9-4468-b9ae-ceb50b5265cc None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:38.823215 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c509b92-47d9-4468-b9ae-ceb50b5265cc None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:38.823215 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c509b92-47d9-4468-b9ae-ceb50b5265cc None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:38.823215 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 206/402] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 11:59:38 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:59:38.823986 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 196/402] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Sep 9 11:59:38 2022] GET /identity/v3/projects/5f8dc54c23f543d197444a43c6b82c01 => generated 302 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:59:38.825887 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793b6de3-3bd7-41d4-93d2-bf1e26ecc694 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:38.825887 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793b6de3-3bd7-41d4-93d2-bf1e26ecc694 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:38.825887 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793b6de3-3bd7-41d4-93d2-bf1e26ecc694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:38.828112 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247faa10-f346-4b1a-b021-ff10be9fd48a None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:38.828629 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247faa10-f346-4b1a-b021-ff10be9fd48a None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:38.829129 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247faa10-f346-4b1a-b021-ff10be9fd48a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:38.848485 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-793b6de3-3bd7-41d4-93d2-bf1e26ecc694 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:59:38.850105 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-247faa10-f346-4b1a-b021-ff10be9fd48a None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:59:38.850729 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-793b6de3-3bd7-41d4-93d2-bf1e26ecc694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:38.851747 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-247faa10-f346-4b1a-b021-ff10be9fd48a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:38.865156 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:38.941677 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 197/403] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 11:59:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 11:59:38.942628 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247faa10-f346-4b1a-b021-ff10be9fd48a None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:38.950181 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 207/404] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 11:59:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:59:38.972165 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:39.070375 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247faa10-f346-4b1a-b021-ff10be9fd48a None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:39.107295 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39533256-2ed2-410d-a015-f2929a297c65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:39.107962 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39533256-2ed2-410d-a015-f2929a297c65 None admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:39.108240 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39533256-2ed2-410d-a015-f2929a297c65 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:39.108507 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39533256-2ed2-410d-a015-f2929a297c65 None admin] PATH_INFO: `/v3/projects/5f8dc54c23f543d197444a43c6b82c01/groups/09faa8b4bc694f2aa6c4a00e0736338e/roles/eecd2f160dd844f5906f3d345780c501` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:39.110437 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39533256-2ed2-410d-a015-f2929a297c65 None admin] RBAC: Authorizing `identity:create_grant(project_id=5f8dc54c23f543d197444a43c6b82c01, group_id=09faa8b4bc694f2aa6c4a00e0736338e, role_id=eecd2f160dd844f5906f3d345780c501)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:39.111917 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39533256-2ed2-410d-a015-f2929a297c65 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:39.129640 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 198/405] 10.222.0.49 () {58 vars in 1442 bytes} [Fri Sep 9 11:59:38 2022] PUT /identity/v3/projects/5f8dc54c23f543d197444a43c6b82c01/groups/09faa8b4bc694f2aa6c4a00e0736338e/roles/eecd2f160dd844f5906f3d345780c501 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:59:39.227638 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793b6de3-3bd7-41d4-93d2-bf1e26ecc694 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:39.265416 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39533256-2ed2-410d-a015-f2929a297c65 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:39.385966 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793b6de3-3bd7-41d4-93d2-bf1e26ecc694 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:39.399897 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:39.427744 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d69fe99-eaaf-4064-8f84-ab889a58b801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:39.428380 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d69fe99-eaaf-4064-8f84-ab889a58b801 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:39.428658 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d69fe99-eaaf-4064-8f84-ab889a58b801 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:39.428927 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d69fe99-eaaf-4064-8f84-ab889a58b801 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:39.436464 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39533256-2ed2-410d-a015-f2929a297c65 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:39.443282 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d69fe99-eaaf-4064-8f84-ab889a58b801 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:59:39.448369 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-5d69fe99-eaaf-4064-8f84-ab889a58b801 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 11:59:39.448369 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:59:39.448369 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:59:39.448369 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:59:39.448369 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:59:39.448369 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:59:39.448369 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:59:39.448369 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:59:39.448369 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:59:39.448369 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:59:39.448369 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:59:39.448369 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:59:39.448369 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:59:39.448369 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:59:39.448369 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:59:39.448369 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:59:39.449267 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:59:39.449267 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:59:39.449267 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:59:39.449267 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:59:39.449267 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:59:39.449267 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:59:39.449267 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:59:39.449267 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:59:39.449267 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:59:39.449267 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:59:39.449267 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:59:39.449267 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 11:59:39.449267 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:59:39.449267 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:59:39.449267 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:59:39.449267 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:59:39.450056 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:59:39.450056 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:59:39.450056 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:59:39.450056 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:59:39.450056 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:59:39.450056 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:59:39.450056 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:59:39.450056 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:59:39.450056 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:59:39.450056 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:59:39.450056 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 11:59:39.450056 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 11:59:39.450056 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 208/406] 10.222.0.49 () {56 vars in 1114 bytes} [Fri Sep 9 11:59:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:59:39.455849 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d69fe99-eaaf-4064-8f84-ab889a58b801 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:39.462203 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:39.495312 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35bfe02-6531-4959-94e2-d227bd7d7c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:39.497406 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35bfe02-6531-4959-94e2-d227bd7d7c94 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:39.498417 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35bfe02-6531-4959-94e2-d227bd7d7c94 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:39.499160 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35bfe02-6531-4959-94e2-d227bd7d7c94 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:39.524232 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35bfe02-6531-4959-94e2-d227bd7d7c94 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:59:39.536140 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-f35bfe02-6531-4959-94e2-d227bd7d7c94 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:59:39.536140 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:59:39.536140 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:59:39.536140 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:59:39.536140 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:59:39.536140 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:59:39.536140 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:59:39.536140 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:59:39.536140 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:59:39.536140 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:59:39.536140 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:59:39.536140 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:59:39.536140 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:59:39.536140 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:59:39.536140 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:59:39.536140 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:59:39.537083 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:59:39.537083 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:59:39.537083 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:59:39.537083 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:59:39.537083 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:59:39.537083 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:59:39.537083 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:59:39.537083 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:59:39.537083 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:59:39.537083 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:59:39.537083 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:59:39.537083 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 11:59:39.537083 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:59:39.537083 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:59:39.537083 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:59:39.537083 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:59:39.537861 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:59:39.537861 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:59:39.537861 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:59:39.537861 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:59:39.537861 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:59:39.537861 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:59:39.537861 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:59:39.537861 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:59:39.537861 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:59:39.537861 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:59:39.537861 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:59:39.537861 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 11:59:39.540535 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 199/407] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Sep 9 11:59:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:59:39.546125 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35bfe02-6531-4959-94e2-d227bd7d7c94 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:39.664264 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d69fe99-eaaf-4064-8f84-ab889a58b801 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:39.711484 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461cb405-6d6c-4cc4-9399-d6fa3f5d7ead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:39.712096 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461cb405-6d6c-4cc4-9399-d6fa3f5d7ead None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:39.712368 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461cb405-6d6c-4cc4-9399-d6fa3f5d7ead None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:39.712634 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461cb405-6d6c-4cc4-9399-d6fa3f5d7ead None admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:39.713301 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461cb405-6d6c-4cc4-9399-d6fa3f5d7ead None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:59:39.713599 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461cb405-6d6c-4cc4-9399-d6fa3f5d7ead None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:39.714989 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461cb405-6d6c-4cc4-9399-d6fa3f5d7ead None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:39.731390 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 209/408] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 9 11:59:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:39.736583 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461cb405-6d6c-4cc4-9399-d6fa3f5d7ead None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:39.751906 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35bfe02-6531-4959-94e2-d227bd7d7c94 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:39.785226 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:39.800215 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57b5b23-c237-439c-8f2f-736d6ea0b078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:39.800835 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57b5b23-c237-439c-8f2f-736d6ea0b078 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:39.801110 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57b5b23-c237-439c-8f2f-736d6ea0b078 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:39.801375 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57b5b23-c237-439c-8f2f-736d6ea0b078 None admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:39.802078 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57b5b23-c237-439c-8f2f-736d6ea0b078 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:59:39.802375 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57b5b23-c237-439c-8f2f-736d6ea0b078 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:39.803769 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57b5b23-c237-439c-8f2f-736d6ea0b078 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:39.826398 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 200/409] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 9 11:59:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:39.829810 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57b5b23-c237-439c-8f2f-736d6ea0b078 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:39.870213 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:39.929611 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461cb405-6d6c-4cc4-9399-d6fa3f5d7ead None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:39.993986 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e43b38f-6b4d-4edd-bbba-b4e5f6807937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:39.995780 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e43b38f-6b4d-4edd-bbba-b4e5f6807937 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:39.996897 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e43b38f-6b4d-4edd-bbba-b4e5f6807937 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:39.997522 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e43b38f-6b4d-4edd-bbba-b4e5f6807937 None admin] PATH_INFO: `/v3/groups/09faa8b4bc694f2aa6c4a00e0736338e` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:39.999952 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e43b38f-6b4d-4edd-bbba-b4e5f6807937 None admin] RBAC: Authorizing `identity:get_group(group_id=09faa8b4bc694f2aa6c4a00e0736338e)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:40.002222 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e43b38f-6b4d-4edd-bbba-b4e5f6807937 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:40.004184 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 210/410] 10.222.0.49 () {56 vars in 1180 bytes} [Fri Sep 9 11:59:39 2022] GET /identity/v3/groups/09faa8b4bc694f2aa6c4a00e0736338e => generated 226 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:59:40.012845 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57b5b23-c237-439c-8f2f-736d6ea0b078 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:40.013971 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e43b38f-6b4d-4edd-bbba-b4e5f6807937 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:40.071882 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6de849-f93b-4421-9f12-45bc5b00115c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:40.072523 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6de849-f93b-4421-9f12-45bc5b00115c None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:40.072885 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6de849-f93b-4421-9f12-45bc5b00115c None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:40.073158 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6de849-f93b-4421-9f12-45bc5b00115c None admin] PATH_INFO: `/v3/groups/1f810b1ba4b7474589f25485737977bb` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:40.077370 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6de849-f93b-4421-9f12-45bc5b00115c None admin] RBAC: Authorizing `identity:get_group(group_id=1f810b1ba4b7474589f25485737977bb)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:40.078950 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6de849-f93b-4421-9f12-45bc5b00115c None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:40.085518 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 201/411] 10.222.0.49 () {56 vars in 1180 bytes} [Fri Sep 9 11:59:39 2022] GET /identity/v3/groups/1f810b1ba4b7474589f25485737977bb => generated 229 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:40.091175 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6de849-f93b-4421-9f12-45bc5b00115c None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:40.202627 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e43b38f-6b4d-4edd-bbba-b4e5f6807937 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:40.219681 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:40.281906 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd149412-58f1-4d1d-b8a2-c7a97422de52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:40.282557 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd149412-58f1-4d1d-b8a2-c7a97422de52 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:40.282835 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd149412-58f1-4d1d-b8a2-c7a97422de52 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:40.283135 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd149412-58f1-4d1d-b8a2-c7a97422de52 None admin] PATH_INFO: `/v3/projects/5f8dc54c23f543d197444a43c6b82c01` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:40.284468 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd149412-58f1-4d1d-b8a2-c7a97422de52 None admin] RBAC: Authorizing `identity:get_project(project_id=5f8dc54c23f543d197444a43c6b82c01)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:40.285954 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd149412-58f1-4d1d-b8a2-c7a97422de52 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:40.288305 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 211/412] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Sep 9 11:59:40 2022] GET /identity/v3/projects/5f8dc54c23f543d197444a43c6b82c01 => generated 302 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:40.298558 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd149412-58f1-4d1d-b8a2-c7a97422de52 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:40.308012 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6de849-f93b-4421-9f12-45bc5b00115c None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:40.311217 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:40.369965 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b7f690-1510-47cd-9f4e-1fa8714c9f62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:40.370658 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b7f690-1510-47cd-9f4e-1fa8714c9f62 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:40.370975 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b7f690-1510-47cd-9f4e-1fa8714c9f62 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:40.371308 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b7f690-1510-47cd-9f4e-1fa8714c9f62 None admin] PATH_INFO: `/v3/projects/ae50911c7cf74f75b5b41fa5275c8468` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:40.372960 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b7f690-1510-47cd-9f4e-1fa8714c9f62 None admin] RBAC: Authorizing `identity:get_project(project_id=ae50911c7cf74f75b5b41fa5275c8468)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:40.374545 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b7f690-1510-47cd-9f4e-1fa8714c9f62 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:40.376541 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 202/413] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Sep 9 11:59:40 2022] GET /identity/v3/projects/ae50911c7cf74f75b5b41fa5275c8468 => generated 344 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:40.382058 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b7f690-1510-47cd-9f4e-1fa8714c9f62 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:40.464285 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd149412-58f1-4d1d-b8a2-c7a97422de52 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:40.506848 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d752ecc-32e0-46d3-b4cd-41a0f524d0a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:40.507595 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d752ecc-32e0-46d3-b4cd-41a0f524d0a4 None admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:40.507973 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d752ecc-32e0-46d3-b4cd-41a0f524d0a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:40.508256 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d752ecc-32e0-46d3-b4cd-41a0f524d0a4 None admin] PATH_INFO: `/v3/projects/5f8dc54c23f543d197444a43c6b82c01/groups/09faa8b4bc694f2aa6c4a00e0736338e/roles/a7c71d2d9914408bb9953f85329d1e80` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:40.510561 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d752ecc-32e0-46d3-b4cd-41a0f524d0a4 None admin] RBAC: Authorizing `identity:create_grant(project_id=5f8dc54c23f543d197444a43c6b82c01, group_id=09faa8b4bc694f2aa6c4a00e0736338e, role_id=a7c71d2d9914408bb9953f85329d1e80)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:40.513056 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d752ecc-32e0-46d3-b4cd-41a0f524d0a4 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:40.517557 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b7f690-1510-47cd-9f4e-1fa8714c9f62 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:40.529850 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 212/414] 10.222.0.49 () {58 vars in 1442 bytes} [Fri Sep 9 11:59:40 2022] PUT /identity/v3/projects/5f8dc54c23f543d197444a43c6b82c01/groups/09faa8b4bc694f2aa6c4a00e0736338e/roles/a7c71d2d9914408bb9953f85329d1e80 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:59:40.532069 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca89626-4159-42ac-800c-3dc639ced0ac None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:40.532641 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca89626-4159-42ac-800c-3dc639ced0ac None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:40.532982 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca89626-4159-42ac-800c-3dc639ced0ac None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:40.540673 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 213/415] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 11:59:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:59:40.543324 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9559a6b4-0fb9-4a8d-9327-1a46febb1770 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:40.543610 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9559a6b4-0fb9-4a8d-9327-1a46febb1770 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:40.543884 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9559a6b4-0fb9-4a8d-9327-1a46febb1770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:40.578099 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:40.585601 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-9559a6b4-0fb9-4a8d-9327-1a46febb1770 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:59:40.586825 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-9559a6b4-0fb9-4a8d-9327-1a46febb1770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:40.595812 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96de3c77-aa37-4b5c-87b4-bf0e895c76f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:40.596511 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96de3c77-aa37-4b5c-87b4-bf0e895c76f3 None admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:40.596790 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96de3c77-aa37-4b5c-87b4-bf0e895c76f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:40.597071 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96de3c77-aa37-4b5c-87b4-bf0e895c76f3 None admin] PATH_INFO: `/v3/projects/ae50911c7cf74f75b5b41fa5275c8468/groups/1f810b1ba4b7474589f25485737977bb/roles/3f2ca33902a04b6dbc2ef7d06d7289d3` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:40.604891 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96de3c77-aa37-4b5c-87b4-bf0e895c76f3 None admin] RBAC: Authorizing `identity:create_grant(project_id=ae50911c7cf74f75b5b41fa5275c8468, group_id=1f810b1ba4b7474589f25485737977bb, role_id=3f2ca33902a04b6dbc2ef7d06d7289d3)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:40.606401 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96de3c77-aa37-4b5c-87b4-bf0e895c76f3 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:40.635260 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 203/416] 10.222.0.49 () {58 vars in 1442 bytes} [Fri Sep 9 11:59:40 2022] PUT /identity/v3/projects/ae50911c7cf74f75b5b41fa5275c8468/groups/1f810b1ba4b7474589f25485737977bb/roles/3f2ca33902a04b6dbc2ef7d06d7289d3 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 11:59:40.692797 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:40.698838 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 214/417] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 11:59:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 11:59:40.981056 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96de3c77-aa37-4b5c-87b4-bf0e895c76f3 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:41.091132 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96de3c77-aa37-4b5c-87b4-bf0e895c76f3 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:41.132670 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97c70ec-5366-4b60-8f8f-2b81d8c5f132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:41.133311 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97c70ec-5366-4b60-8f8f-2b81d8c5f132 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:41.133592 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97c70ec-5366-4b60-8f8f-2b81d8c5f132 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:41.133867 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97c70ec-5366-4b60-8f8f-2b81d8c5f132 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:41.151747 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97c70ec-5366-4b60-8f8f-2b81d8c5f132 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:59:41.157473 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-a97c70ec-5366-4b60-8f8f-2b81d8c5f132 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 11:59:41.157473 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:59:41.157473 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:59:41.157473 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:59:41.157473 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:59:41.157473 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:59:41.157473 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:59:41.157473 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:59:41.157473 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:59:41.157473 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:59:41.157473 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:59:41.157473 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:59:41.157473 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:59:41.157473 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:59:41.157473 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:59:41.157473 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:59:41.158438 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:59:41.158438 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:59:41.158438 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:59:41.158438 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:59:41.158438 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:59:41.158438 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:59:41.158438 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:59:41.158438 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:59:41.158438 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:59:41.158438 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:59:41.158438 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:59:41.158438 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 11:59:41.158438 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:59:41.158438 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:59:41.158438 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:59:41.158438 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:59:41.160244 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:59:41.160244 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:59:41.160244 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:59:41.160244 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:59:41.160244 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:59:41.160244 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:59:41.160244 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:59:41.160244 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:59:41.160244 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:59:41.160244 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:59:41.160244 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 11:59:41.160244 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 11:59:41.160244 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 204/418] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 9 11:59:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:59:41.165108 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9559a6b4-0fb9-4a8d-9327-1a46febb1770 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:41.277103 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e44b71-3e35-4e30-b84e-3efa1ed90234 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:41.277437 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e44b71-3e35-4e30-b84e-3efa1ed90234 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:41.277659 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e44b71-3e35-4e30-b84e-3efa1ed90234 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:41.279162 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 205/419] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 11:59:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 11:59:41.284238 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f524c2d-9602-43d5-a1d8-ea0b7eb1638a None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:41.284524 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f524c2d-9602-43d5-a1d8-ea0b7eb1638a None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:41.284794 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f524c2d-9602-43d5-a1d8-ea0b7eb1638a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:41.315877 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-5f524c2d-9602-43d5-a1d8-ea0b7eb1638a None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:59:41.317624 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-5f524c2d-9602-43d5-a1d8-ea0b7eb1638a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:41.322554 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9559a6b4-0fb9-4a8d-9327-1a46febb1770 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:41.386275 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89b6a73-1ce8-4730-9888-aa689525465b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:41.386888 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89b6a73-1ce8-4730-9888-aa689525465b None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:41.387191 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89b6a73-1ce8-4730-9888-aa689525465b None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:41.387458 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89b6a73-1ce8-4730-9888-aa689525465b None admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:41.388124 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89b6a73-1ce8-4730-9888-aa689525465b None admin] RBAC: Adding query filter params (name=member) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:59:41.388423 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89b6a73-1ce8-4730-9888-aa689525465b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:41.389803 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89b6a73-1ce8-4730-9888-aa689525465b None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:41.409419 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 215/420] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Sep 9 11:59:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:41.410021 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:41.415248 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89b6a73-1ce8-4730-9888-aa689525465b None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:41.440934 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 206/421] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 11:59:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 11:59:41.521850 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:41.592653 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89b6a73-1ce8-4730-9888-aa689525465b None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:41.637305 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d5d296-ca6b-4a5f-bee6-976566333434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:41.638288 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d5d296-ca6b-4a5f-bee6-976566333434 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:41.638756 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d5d296-ca6b-4a5f-bee6-976566333434 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:41.639280 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d5d296-ca6b-4a5f-bee6-976566333434 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:41.659723 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5191cf-9403-4b0b-92d3-cf0eb86e297c None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:41.660028 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5191cf-9403-4b0b-92d3-cf0eb86e297c None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:41.660275 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5191cf-9403-4b0b-92d3-cf0eb86e297c None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:41.662600 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 207/422] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 11:59:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 11:59:41.680702 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76441750-85fd-49f5-bb75-30303a12c0d8 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:41.680975 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76441750-85fd-49f5-bb75-30303a12c0d8 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:41.681244 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76441750-85fd-49f5-bb75-30303a12c0d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:41.712744 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d5d296-ca6b-4a5f-bee6-976566333434 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:41.714203 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d5d296-ca6b-4a5f-bee6-976566333434 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:41.731993 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-76441750-85fd-49f5-bb75-30303a12c0d8 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:59:41.733231 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-76441750-85fd-49f5-bb75-30303a12c0d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:41.771467 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:41.824203 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 216/423] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 9 11:59:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 09 11:59:41.825166 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d5d296-ca6b-4a5f-bee6-976566333434 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:41.837178 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 208/424] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 11:59:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:59:41.838462 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76441750-85fd-49f5-bb75-30303a12c0d8 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:41.933275 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d5d296-ca6b-4a5f-bee6-976566333434 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:41.970657 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fd8188-3fda-4217-941a-93635fa58d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:41.971587 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fd8188-3fda-4217-941a-93635fa58d0f None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:41.971924 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fd8188-3fda-4217-941a-93635fa58d0f None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:41.972258 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fd8188-3fda-4217-941a-93635fa58d0f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:41.972508 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76441750-85fd-49f5-bb75-30303a12c0d8 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:41.984384 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:41.985151 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fd8188-3fda-4217-941a-93635fa58d0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:59:41.992327 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-61fd8188-3fda-4217-941a-93635fa58d0f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 11:59:41.992327 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:59:41.992327 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:59:41.992327 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:59:41.992327 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:59:41.992327 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:59:41.992327 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:59:41.992327 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:59:41.992327 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:59:41.992327 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:59:41.992327 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:59:41.992327 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:59:41.992327 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:59:41.992327 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:59:41.992327 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:59:41.992327 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:59:41.993472 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:59:41.993472 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:59:41.993472 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:59:41.993472 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:59:41.993472 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:59:41.993472 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:59:41.993472 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:59:41.993472 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:59:41.993472 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:59:41.993472 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:59:41.993472 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:59:41.993472 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 11:59:41.993472 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:59:41.993472 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:59:41.993472 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:59:41.993472 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:59:41.994461 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:59:41.994461 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:59:41.994461 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:59:41.994461 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:59:41.994461 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:59:41.994461 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:59:41.994461 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:59:41.994461 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:59:41.994461 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:59:41.994461 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:59:41.994461 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 11:59:41.994461 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 11:59:41.999072 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 217/425] 10.222.0.49 () {56 vars in 1114 bytes} [Fri Sep 9 11:59:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:59:42.005423 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fd8188-3fda-4217-941a-93635fa58d0f None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:42.019422 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f1901b-6a31-42a4-901d-bee9c98722c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:42.020120 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f1901b-6a31-42a4-901d-bee9c98722c4 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:42.021013 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f1901b-6a31-42a4-901d-bee9c98722c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:42.021451 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f1901b-6a31-42a4-901d-bee9c98722c4 None admin] PATH_INFO: `/v3/projects/f8cf6274da2e456eb20ee0ba0d187a83` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:42.022891 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f1901b-6a31-42a4-901d-bee9c98722c4 None admin] RBAC: Authorizing `identity:get_project(project_id=f8cf6274da2e456eb20ee0ba0d187a83)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:42.024157 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f1901b-6a31-42a4-901d-bee9c98722c4 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:42.026117 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 209/426] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Sep 9 11:59:41 2022] GET /identity/v3/projects/f8cf6274da2e456eb20ee0ba0d187a83 => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:42.026995 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:42.032165 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f1901b-6a31-42a4-901d-bee9c98722c4 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:42.182167 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fd8188-3fda-4217-941a-93635fa58d0f None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:42.222711 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f1901b-6a31-42a4-901d-bee9c98722c4 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:42.248642 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf2bea3-49cc-4906-9169-bda55e5344f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:42.249246 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf2bea3-49cc-4906-9169-bda55e5344f0 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:42.249521 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf2bea3-49cc-4906-9169-bda55e5344f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:42.249788 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf2bea3-49cc-4906-9169-bda55e5344f0 None admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:42.250512 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf2bea3-49cc-4906-9169-bda55e5344f0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:59:42.250777 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf2bea3-49cc-4906-9169-bda55e5344f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:42.252133 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf2bea3-49cc-4906-9169-bda55e5344f0 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:42.263499 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 218/427] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 9 11:59:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:42.263499 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf2bea3-49cc-4906-9169-bda55e5344f0 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:42.282411 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e261f1-3dcb-4b24-8db2-65d9a491378d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:42.283620 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e261f1-3dcb-4b24-8db2-65d9a491378d None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:42.284632 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e261f1-3dcb-4b24-8db2-65d9a491378d None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:42.285086 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e261f1-3dcb-4b24-8db2-65d9a491378d None admin] PATH_INFO: `/v3/groups/09faa8b4bc694f2aa6c4a00e0736338e` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:42.287581 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e261f1-3dcb-4b24-8db2-65d9a491378d None admin] RBAC: Authorizing `identity:get_group(group_id=09faa8b4bc694f2aa6c4a00e0736338e)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:42.289594 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e261f1-3dcb-4b24-8db2-65d9a491378d None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:42.292208 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 210/428] 10.222.0.49 () {56 vars in 1180 bytes} [Fri Sep 9 11:59:42 2022] GET /identity/v3/groups/09faa8b4bc694f2aa6c4a00e0736338e => generated 226 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:42.293499 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e261f1-3dcb-4b24-8db2-65d9a491378d None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:42.357134 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:42.364647 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:42.396423 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf2bea3-49cc-4906-9169-bda55e5344f0 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:42.436473 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e261f1-3dcb-4b24-8db2-65d9a491378d None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:42.440181 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3dd444-f6e3-4cf8-b9fd-5c5434d07812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:42.441073 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3dd444-f6e3-4cf8-b9fd-5c5434d07812 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:42.441550 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3dd444-f6e3-4cf8-b9fd-5c5434d07812 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:42.442029 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3dd444-f6e3-4cf8-b9fd-5c5434d07812 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:42.456812 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3dd444-f6e3-4cf8-b9fd-5c5434d07812 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:59:42.464874 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-0b3dd444-f6e3-4cf8-b9fd-5c5434d07812 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 11:59:42.464874 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:59:42.464874 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:59:42.464874 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:59:42.464874 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:59:42.464874 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:59:42.464874 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:59:42.464874 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:59:42.464874 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:59:42.464874 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:59:42.464874 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:59:42.464874 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:59:42.464874 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:59:42.464874 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:59:42.464874 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:59:42.464874 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:59:42.465752 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:59:42.465752 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:59:42.465752 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:59:42.465752 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:59:42.465752 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:59:42.465752 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:59:42.465752 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:59:42.465752 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:59:42.465752 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:59:42.465752 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:59:42.465752 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:59:42.465752 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 11:59:42.465752 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:59:42.465752 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:59:42.465752 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:59:42.465752 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:59:42.466523 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:59:42.466523 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:59:42.466523 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:59:42.466523 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:59:42.466523 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:59:42.466523 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:59:42.466523 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:59:42.466523 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:59:42.466523 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:59:42.466523 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:59:42.466523 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 11:59:42.466523 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 11:59:42.468269 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 219/429] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 9 11:59:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:59:42.468891 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3dd444-f6e3-4cf8-b9fd-5c5434d07812 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:42.479502 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa0d586-6fe1-498d-9f4c-272883334e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:42.480041 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa0d586-6fe1-498d-9f4c-272883334e68 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:42.480319 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa0d586-6fe1-498d-9f4c-272883334e68 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:42.480589 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa0d586-6fe1-498d-9f4c-272883334e68 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:42.577259 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa0d586-6fe1-498d-9f4c-272883334e68 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:42.577259 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa0d586-6fe1-498d-9f4c-272883334e68 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:42.604701 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3dd444-f6e3-4cf8-b9fd-5c5434d07812 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:42.626368 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:42.641823 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ed2357-ac27-41f1-95c1-f94e0ef79a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:42.642765 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ed2357-ac27-41f1-95c1-f94e0ef79a01 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:42.643274 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ed2357-ac27-41f1-95c1-f94e0ef79a01 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:42.643751 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ed2357-ac27-41f1-95c1-f94e0ef79a01 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:42.644963 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ed2357-ac27-41f1-95c1-f94e0ef79a01 None admin] RBAC: Adding query filter params (group.id=09faa8b4bc694f2aa6c4a00e0736338e, role.id=eecd2f160dd844f5906f3d345780c501, scope.project.id=f8cf6274da2e456eb20ee0ba0d187a83) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:59:42.645652 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ed2357-ac27-41f1-95c1-f94e0ef79a01 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:42.647452 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ed2357-ac27-41f1-95c1-f94e0ef79a01 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:42.651630 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:42.655031 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 220/430] 10.222.0.49 () {56 vars in 1376 bytes} [Fri Sep 9 11:59:42 2022] GET /identity/v3/role_assignments?group.id=09faa8b4bc694f2aa6c4a00e0736338e&scope.project.id=f8cf6274da2e456eb20ee0ba0d187a83&role.id=eecd2f160dd844f5906f3d345780c501 => generated 611 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:42.656066 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ed2357-ac27-41f1-95c1-f94e0ef79a01 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:42.724846 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 211/431] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 9 11:59:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 11:59:42.730362 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa0d586-6fe1-498d-9f4c-272883334e68 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:42.767248 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ed2357-ac27-41f1-95c1-f94e0ef79a01 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:42.806153 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3997b5-2ad1-4bc2-916f-fae03b870109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:42.807246 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3997b5-2ad1-4bc2-916f-fae03b870109 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:42.807785 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3997b5-2ad1-4bc2-916f-fae03b870109 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:42.808258 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3997b5-2ad1-4bc2-916f-fae03b870109 None admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:42.809191 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3997b5-2ad1-4bc2-916f-fae03b870109 None admin] RBAC: Adding query filter params (name=member) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:59:42.809707 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3997b5-2ad1-4bc2-916f-fae03b870109 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:42.811426 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3997b5-2ad1-4bc2-916f-fae03b870109 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:42.827064 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 221/432] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Sep 9 11:59:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:42.832476 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3997b5-2ad1-4bc2-916f-fae03b870109 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:42.942269 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:42.946368 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa0d586-6fe1-498d-9f4c-272883334e68 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:42.997870 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3997b5-2ad1-4bc2-916f-fae03b870109 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:43.008268 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9281b3b8-e5bc-4764-b49e-e9b965956a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:43.009599 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9281b3b8-e5bc-4764-b49e-e9b965956a77 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:43.010080 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9281b3b8-e5bc-4764-b49e-e9b965956a77 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:43.010548 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9281b3b8-e5bc-4764-b49e-e9b965956a77 None admin] PATH_INFO: `/v3/projects/f8cf6274da2e456eb20ee0ba0d187a83` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:43.012028 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9281b3b8-e5bc-4764-b49e-e9b965956a77 None admin] RBAC: Authorizing `identity:get_project(project_id=f8cf6274da2e456eb20ee0ba0d187a83)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:43.014538 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9281b3b8-e5bc-4764-b49e-e9b965956a77 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:43.017773 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 212/433] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Sep 9 11:59:42 2022] GET /identity/v3/projects/f8cf6274da2e456eb20ee0ba0d187a83 => generated 298 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:43.023880 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9281b3b8-e5bc-4764-b49e-e9b965956a77 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:43.043414 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:43.046979 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f767272e-b303-40a6-84b3-4bd10bdffa6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:43.047608 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f767272e-b303-40a6-84b3-4bd10bdffa6f None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:43.047887 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f767272e-b303-40a6-84b3-4bd10bdffa6f None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:43.048158 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f767272e-b303-40a6-84b3-4bd10bdffa6f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:43.126313 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f767272e-b303-40a6-84b3-4bd10bdffa6f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:43.127696 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f767272e-b303-40a6-84b3-4bd10bdffa6f None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:43.152768 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9281b3b8-e5bc-4764-b49e-e9b965956a77 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:43.191443 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44db2ed5-857a-4c45-93ee-718f15ac10f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:43.192869 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:43.193958 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44db2ed5-857a-4c45-93ee-718f15ac10f5 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:43.194439 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44db2ed5-857a-4c45-93ee-718f15ac10f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:43.194969 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44db2ed5-857a-4c45-93ee-718f15ac10f5 None admin] PATH_INFO: `/v3/groups/09faa8b4bc694f2aa6c4a00e0736338e` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:43.196578 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44db2ed5-857a-4c45-93ee-718f15ac10f5 None admin] RBAC: Authorizing `identity:get_group(group_id=09faa8b4bc694f2aa6c4a00e0736338e)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:43.198272 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44db2ed5-857a-4c45-93ee-718f15ac10f5 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:43.200407 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 213/434] 10.222.0.49 () {56 vars in 1180 bytes} [Fri Sep 9 11:59:43 2022] GET /identity/v3/groups/09faa8b4bc694f2aa6c4a00e0736338e => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:43.211947 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44db2ed5-857a-4c45-93ee-718f15ac10f5 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:43.248787 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 222/435] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 9 11:59:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 11:59:43.249884 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5a5e57-1815-43a7-aa0e-4819c80d1368 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:43.250368 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5a5e57-1815-43a7-aa0e-4819c80d1368 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:43.250651 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5a5e57-1815-43a7-aa0e-4819c80d1368 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:43.255096 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 223/436] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 11:59:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:59:43.255910 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5a5e57-1815-43a7-aa0e-4819c80d1368 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:43.321672 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:43.346674 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44db2ed5-857a-4c45-93ee-718f15ac10f5 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:43.389286 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5a5e57-1815-43a7-aa0e-4819c80d1368 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:43.402974 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de28e7cb-bd4e-425a-af6c-059782d166c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:43.403598 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de28e7cb-bd4e-425a-af6c-059782d166c0 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:43.403874 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de28e7cb-bd4e-425a-af6c-059782d166c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:43.404145 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de28e7cb-bd4e-425a-af6c-059782d166c0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:43.404868 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de28e7cb-bd4e-425a-af6c-059782d166c0 None admin] RBAC: Adding query filter params (group.id=09faa8b4bc694f2aa6c4a00e0736338e, role.id=a7c71d2d9914408bb9953f85329d1e80, scope.project.id=f8cf6274da2e456eb20ee0ba0d187a83) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:59:43.405172 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de28e7cb-bd4e-425a-af6c-059782d166c0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:43.406610 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de28e7cb-bd4e-425a-af6c-059782d166c0 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:43.418091 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 214/437] 10.222.0.49 () {56 vars in 1376 bytes} [Fri Sep 9 11:59:43 2022] GET /identity/v3/role_assignments?group.id=09faa8b4bc694f2aa6c4a00e0736338e&scope.project.id=f8cf6274da2e456eb20ee0ba0d187a83&role.id=a7c71d2d9914408bb9953f85329d1e80 => generated 611 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:43.419966 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a67d30-8f35-4df7-8971-0fb2a95213f1 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:43.419966 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a67d30-8f35-4df7-8971-0fb2a95213f1 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:43.420132 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a67d30-8f35-4df7-8971-0fb2a95213f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:43.433033 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb2e958-1efc-46d2-a533-422509169a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:43.434713 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb2e958-1efc-46d2-a533-422509169a71 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:43.435396 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb2e958-1efc-46d2-a533-422509169a71 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:43.435974 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb2e958-1efc-46d2-a533-422509169a71 None admin] PATH_INFO: `/v3/projects/5f8dc54c23f543d197444a43c6b82c01` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:43.437432 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb2e958-1efc-46d2-a533-422509169a71 None admin] RBAC: Authorizing `identity:get_project(project_id=5f8dc54c23f543d197444a43c6b82c01)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:43.440072 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb2e958-1efc-46d2-a533-422509169a71 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:43.440261 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-c2a67d30-8f35-4df7-8971-0fb2a95213f1 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:59:43.441601 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-c2a67d30-8f35-4df7-8971-0fb2a95213f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:43.443332 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 224/438] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Sep 9 11:59:43 2022] GET /identity/v3/projects/5f8dc54c23f543d197444a43c6b82c01 => generated 302 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:43.449208 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb2e958-1efc-46d2-a533-422509169a71 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:43.496496 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:43.543333 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 215/439] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 11:59:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:59:43.544655 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c600c5-af8d-4338-8c75-ed303e1f41d7 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:43.544946 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c600c5-af8d-4338-8c75-ed303e1f41d7 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:43.545212 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c600c5-af8d-4338-8c75-ed303e1f41d7 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:43.550865 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 216/440] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 11:59:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:59:43.556100 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc30839-c757-4579-aa9f-a88eeff5faac None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:43.556771 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc30839-c757-4579-aa9f-a88eeff5faac None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:43.557218 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc30839-c757-4579-aa9f-a88eeff5faac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:43.578536 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-ecc30839-c757-4579-aa9f-a88eeff5faac None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:59:43.580345 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-ecc30839-c757-4579-aa9f-a88eeff5faac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:43.584049 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb2e958-1efc-46d2-a533-422509169a71 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:43.622859 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f491b104-8e97-46c7-9672-7f47809ed38f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:43.623576 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f491b104-8e97-46c7-9672-7f47809ed38f None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:43.623834 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f491b104-8e97-46c7-9672-7f47809ed38f None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:43.624106 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f491b104-8e97-46c7-9672-7f47809ed38f None admin] PATH_INFO: `/v3/groups/09faa8b4bc694f2aa6c4a00e0736338e` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:43.626623 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f491b104-8e97-46c7-9672-7f47809ed38f None admin] RBAC: Authorizing `identity:get_group(group_id=09faa8b4bc694f2aa6c4a00e0736338e)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:43.627698 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f491b104-8e97-46c7-9672-7f47809ed38f None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:43.629316 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 225/441] 10.222.0.49 () {56 vars in 1180 bytes} [Fri Sep 9 11:59:43 2022] GET /identity/v3/groups/09faa8b4bc694f2aa6c4a00e0736338e => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:43.636022 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f491b104-8e97-46c7-9672-7f47809ed38f None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:43.674072 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 217/442] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 11:59:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:59:43.746807 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:43.751078 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f491b104-8e97-46c7-9672-7f47809ed38f None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:43.790978 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb48794-4bc9-462d-83dc-dc6444818e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:43.791589 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb48794-4bc9-462d-83dc-dc6444818e3e None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:43.791865 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb48794-4bc9-462d-83dc-dc6444818e3e None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:43.792134 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb48794-4bc9-462d-83dc-dc6444818e3e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:43.792857 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb48794-4bc9-462d-83dc-dc6444818e3e None admin] RBAC: Adding query filter params (group.id=09faa8b4bc694f2aa6c4a00e0736338e, role.id=eecd2f160dd844f5906f3d345780c501, scope.project.id=5f8dc54c23f543d197444a43c6b82c01) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:59:43.793161 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb48794-4bc9-462d-83dc-dc6444818e3e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:43.794637 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb48794-4bc9-462d-83dc-dc6444818e3e None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:43.801085 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 226/443] 10.222.0.49 () {56 vars in 1376 bytes} [Fri Sep 9 11:59:43 2022] GET /identity/v3/role_assignments?group.id=09faa8b4bc694f2aa6c4a00e0736338e&scope.project.id=5f8dc54c23f543d197444a43c6b82c01&role.id=eecd2f160dd844f5906f3d345780c501 => generated 611 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:43.826554 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc30839-c757-4579-aa9f-a88eeff5faac None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:43.831288 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:43.949182 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc30839-c757-4579-aa9f-a88eeff5faac None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:43.960109 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb48794-4bc9-462d-83dc-dc6444818e3e None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:43.995316 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fc3982-a500-469a-80a9-9585ee4fc183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:43.996022 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fc3982-a500-469a-80a9-9585ee4fc183 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:43.996296 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fc3982-a500-469a-80a9-9585ee4fc183 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:43.996571 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fc3982-a500-469a-80a9-9585ee4fc183 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:44.006952 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fc3982-a500-469a-80a9-9585ee4fc183 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:59:44.011856 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-71fc3982-a500-469a-80a9-9585ee4fc183 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 11:59:44.011856 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:59:44.011856 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:59:44.011856 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:59:44.011856 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:59:44.011856 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:59:44.011856 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:59:44.011856 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:59:44.011856 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:59:44.011856 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:59:44.011856 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:59:44.011856 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:59:44.011856 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:59:44.011856 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:59:44.011856 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:59:44.011856 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:59:44.013216 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:59:44.013216 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:59:44.013216 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:59:44.013216 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:59:44.013216 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:59:44.013216 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:59:44.013216 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:59:44.013216 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:59:44.013216 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:59:44.013216 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:59:44.013216 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:59:44.013216 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 11:59:44.013216 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:59:44.013216 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:59:44.013216 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:59:44.013216 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:59:44.014005 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:59:44.014005 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:59:44.014005 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:59:44.014005 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:59:44.014005 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:59:44.014005 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:59:44.014005 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:59:44.014005 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:59:44.014005 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:59:44.014005 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:59:44.014005 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 11:59:44.014005 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 11:59:44.015104 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 218/444] 10.222.0.49 () {56 vars in 1114 bytes} [Fri Sep 9 11:59:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:59:44.019146 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fc3982-a500-469a-80a9-9585ee4fc183 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:44.080858 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb48794-4bc9-462d-83dc-dc6444818e3e None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:44.092113 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:44.120787 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067fcd7f-8f1c-42d0-beb6-7973970fa8c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:44.121723 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067fcd7f-8f1c-42d0-beb6-7973970fa8c1 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:44.122016 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067fcd7f-8f1c-42d0-beb6-7973970fa8c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:44.122290 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067fcd7f-8f1c-42d0-beb6-7973970fa8c1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:44.136241 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067fcd7f-8f1c-42d0-beb6-7973970fa8c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:59:44.141106 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fc3982-a500-469a-80a9-9585ee4fc183 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:44.141641 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-067fcd7f-8f1c-42d0-beb6-7973970fa8c1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:59:44.141641 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:59:44.141641 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:59:44.141641 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:59:44.141641 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:59:44.141641 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:59:44.141641 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:59:44.141641 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:59:44.141641 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:59:44.141641 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:59:44.141641 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:59:44.141641 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:59:44.141641 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:59:44.141641 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:59:44.141641 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:59:44.141641 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:59:44.142674 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:59:44.142674 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:59:44.142674 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:59:44.142674 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:59:44.142674 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:59:44.142674 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:59:44.142674 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:59:44.142674 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:59:44.142674 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:59:44.142674 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:59:44.142674 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:59:44.142674 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 11:59:44.142674 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:59:44.142674 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:59:44.142674 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:59:44.142674 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:59:44.143546 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:59:44.143546 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:59:44.143546 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:59:44.143546 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:59:44.143546 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:59:44.143546 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:59:44.143546 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:59:44.143546 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:59:44.143546 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:59:44.143546 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:59:44.143546 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 11:59:44.143546 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 11:59:44.143546 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 227/445] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Sep 9 11:59:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:59:44.150341 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067fcd7f-8f1c-42d0-beb6-7973970fa8c1 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:44.181012 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:44.192486 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409fe8da-e460-49e8-b9f6-9e8ee266498b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:44.193218 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409fe8da-e460-49e8-b9f6-9e8ee266498b None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:44.193494 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409fe8da-e460-49e8-b9f6-9e8ee266498b None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:44.194510 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409fe8da-e460-49e8-b9f6-9e8ee266498b None admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:44.194510 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409fe8da-e460-49e8-b9f6-9e8ee266498b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:59:44.194819 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409fe8da-e460-49e8-b9f6-9e8ee266498b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:44.196263 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409fe8da-e460-49e8-b9f6-9e8ee266498b None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:44.206941 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 219/446] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 9 11:59:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:59:44.213566 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409fe8da-e460-49e8-b9f6-9e8ee266498b None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:44.265179 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067fcd7f-8f1c-42d0-beb6-7973970fa8c1 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:44.297684 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac6fece-ca4b-4d6d-ba5e-30c4441348ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:44.298276 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac6fece-ca4b-4d6d-ba5e-30c4441348ef None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:44.298548 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac6fece-ca4b-4d6d-ba5e-30c4441348ef None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:44.298814 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac6fece-ca4b-4d6d-ba5e-30c4441348ef None admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:44.299488 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac6fece-ca4b-4d6d-ba5e-30c4441348ef None admin] RBAC: Adding query filter params (name=admin) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:59:44.299789 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac6fece-ca4b-4d6d-ba5e-30c4441348ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:44.301171 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac6fece-ca4b-4d6d-ba5e-30c4441348ef None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:44.309099 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 228/447] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 9 11:59:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:59:44.314346 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409fe8da-e460-49e8-b9f6-9e8ee266498b None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:44.314933 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac6fece-ca4b-4d6d-ba5e-30c4441348ef None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:44.340825 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:44.349915 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe87224-1124-48de-834d-95c914495e0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:44.350526 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe87224-1124-48de-834d-95c914495e0f None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:44.350806 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe87224-1124-48de-834d-95c914495e0f None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:44.351104 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe87224-1124-48de-834d-95c914495e0f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:44.398822 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:44.414521 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac6fece-ca4b-4d6d-ba5e-30c4441348ef None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:44.419221 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe87224-1124-48de-834d-95c914495e0f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:44.420679 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe87224-1124-48de-834d-95c914495e0f None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:44.448782 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6403af34-37b2-4c56-9bfd-f3609714e013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:44.449379 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6403af34-37b2-4c56-9bfd-f3609714e013 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:44.449661 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6403af34-37b2-4c56-9bfd-f3609714e013 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:44.449928 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6403af34-37b2-4c56-9bfd-f3609714e013 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:44.516125 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6403af34-37b2-4c56-9bfd-f3609714e013 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:44.517561 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6403af34-37b2-4c56-9bfd-f3609714e013 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:44.521790 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 220/448] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 9 11:59:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 11:59:44.527701 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe87224-1124-48de-834d-95c914495e0f None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:44.553122 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:44.600677 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:44.617903 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 229/449] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 9 11:59:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 09 11:59:44.623597 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6403af34-37b2-4c56-9bfd-f3609714e013 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:44.627604 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe87224-1124-48de-834d-95c914495e0f None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:44.660310 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4664a8-335b-4d44-aa1e-83a1459a2909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:44.661065 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4664a8-335b-4d44-aa1e-83a1459a2909 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:44.661431 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4664a8-335b-4d44-aa1e-83a1459a2909 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:44.661707 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4664a8-335b-4d44-aa1e-83a1459a2909 None admin] PATH_INFO: `/v3/projects/5f8dc54c23f543d197444a43c6b82c01` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:44.662837 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4664a8-335b-4d44-aa1e-83a1459a2909 None admin] RBAC: Authorizing `identity:get_project(project_id=5f8dc54c23f543d197444a43c6b82c01)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:44.664297 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4664a8-335b-4d44-aa1e-83a1459a2909 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:44.666202 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 221/450] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Sep 9 11:59:44 2022] GET /identity/v3/projects/5f8dc54c23f543d197444a43c6b82c01 => generated 302 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:44.671575 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4664a8-335b-4d44-aa1e-83a1459a2909 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:44.727979 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6403af34-37b2-4c56-9bfd-f3609714e013 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:44.762770 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:44.762979 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e48701-5dce-40c4-a914-07180fca1345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:44.763581 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e48701-5dce-40c4-a914-07180fca1345 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:44.763959 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e48701-5dce-40c4-a914-07180fca1345 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:44.764123 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e48701-5dce-40c4-a914-07180fca1345 None admin] PATH_INFO: `/v3/projects/ae50911c7cf74f75b5b41fa5275c8468` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:44.765419 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e48701-5dce-40c4-a914-07180fca1345 None admin] RBAC: Authorizing `identity:get_project(project_id=ae50911c7cf74f75b5b41fa5275c8468)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:44.767115 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e48701-5dce-40c4-a914-07180fca1345 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:44.768919 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 230/451] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Sep 9 11:59:44 2022] GET /identity/v3/projects/ae50911c7cf74f75b5b41fa5275c8468 => generated 344 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:59:44.770017 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4664a8-335b-4d44-aa1e-83a1459a2909 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:44.775058 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e48701-5dce-40c4-a914-07180fca1345 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:44.804695 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb085b0-a135-47dc-9e37-52f3ab2fa7b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:44.805320 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb085b0-a135-47dc-9e37-52f3ab2fa7b0 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:44.805830 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb085b0-a135-47dc-9e37-52f3ab2fa7b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:44.805830 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb085b0-a135-47dc-9e37-52f3ab2fa7b0 None admin] PATH_INFO: `/v3/groups/09faa8b4bc694f2aa6c4a00e0736338e` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:44.806981 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb085b0-a135-47dc-9e37-52f3ab2fa7b0 None admin] RBAC: Authorizing `identity:get_group(group_id=09faa8b4bc694f2aa6c4a00e0736338e)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:44.808483 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb085b0-a135-47dc-9e37-52f3ab2fa7b0 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:44.810107 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 222/452] 10.222.0.49 () {56 vars in 1180 bytes} [Fri Sep 9 11:59:44 2022] GET /identity/v3/groups/09faa8b4bc694f2aa6c4a00e0736338e => generated 226 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:59:44.816071 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb085b0-a135-47dc-9e37-52f3ab2fa7b0 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:44.827214 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:44.874042 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e48701-5dce-40c4-a914-07180fca1345 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:44.907524 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e07fc8-bc2c-4b18-ba9c-422939074a90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:44.908146 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e07fc8-bc2c-4b18-ba9c-422939074a90 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:44.908423 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e07fc8-bc2c-4b18-ba9c-422939074a90 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:44.908692 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e07fc8-bc2c-4b18-ba9c-422939074a90 None admin] PATH_INFO: `/v3/groups/1f810b1ba4b7474589f25485737977bb` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:44.909779 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e07fc8-bc2c-4b18-ba9c-422939074a90 None admin] RBAC: Authorizing `identity:get_group(group_id=1f810b1ba4b7474589f25485737977bb)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:44.912069 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e07fc8-bc2c-4b18-ba9c-422939074a90 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:44.913675 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 231/453] 10.222.0.49 () {56 vars in 1180 bytes} [Fri Sep 9 11:59:44 2022] GET /identity/v3/groups/1f810b1ba4b7474589f25485737977bb => generated 229 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:59:44.919508 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e07fc8-bc2c-4b18-ba9c-422939074a90 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:44.919821 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb085b0-a135-47dc-9e37-52f3ab2fa7b0 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:44.953588 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b86b38-cced-483d-8f4d-2fc28650b00a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:44.954185 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b86b38-cced-483d-8f4d-2fc28650b00a None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:44.954461 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b86b38-cced-483d-8f4d-2fc28650b00a None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:44.954729 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b86b38-cced-483d-8f4d-2fc28650b00a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:44.955547 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b86b38-cced-483d-8f4d-2fc28650b00a None admin] RBAC: Adding query filter params (group.id=09faa8b4bc694f2aa6c4a00e0736338e, role.id=a7c71d2d9914408bb9953f85329d1e80, scope.project.id=5f8dc54c23f543d197444a43c6b82c01) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:59:44.955855 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b86b38-cced-483d-8f4d-2fc28650b00a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:44.957286 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b86b38-cced-483d-8f4d-2fc28650b00a None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:44.963680 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 223/454] 10.222.0.49 () {56 vars in 1376 bytes} [Fri Sep 9 11:59:44 2022] GET /identity/v3/role_assignments?group.id=09faa8b4bc694f2aa6c4a00e0736338e&scope.project.id=5f8dc54c23f543d197444a43c6b82c01&role.id=a7c71d2d9914408bb9953f85329d1e80 => generated 611 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:59:45.023122 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e07fc8-bc2c-4b18-ba9c-422939074a90 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:45.045193 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:45.060478 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0c2d3a-3922-4ac4-a342-93fc53479933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:45.061091 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0c2d3a-3922-4ac4-a342-93fc53479933 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:45.061369 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0c2d3a-3922-4ac4-a342-93fc53479933 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:45.061640 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0c2d3a-3922-4ac4-a342-93fc53479933 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:45.062374 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0c2d3a-3922-4ac4-a342-93fc53479933 None admin] RBAC: Adding query filter params (group.id=1f810b1ba4b7474589f25485737977bb, role.id=3f2ca33902a04b6dbc2ef7d06d7289d3, scope.project.id=ae50911c7cf74f75b5b41fa5275c8468) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:59:45.062678 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0c2d3a-3922-4ac4-a342-93fc53479933 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:45.064175 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0c2d3a-3922-4ac4-a342-93fc53479933 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:45.072478 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 232/455] 10.222.0.49 () {56 vars in 1376 bytes} [Fri Sep 9 11:59:44 2022] GET /identity/v3/role_assignments?group.id=1f810b1ba4b7474589f25485737977bb&scope.project.id=ae50911c7cf74f75b5b41fa5275c8468&role.id=3f2ca33902a04b6dbc2ef7d06d7289d3 => generated 611 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:59:47.708465 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d539898c-2ba0-446d-b0ef-c9843d745a70 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:47.708972 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d539898c-2ba0-446d-b0ef-c9843d745a70 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:47.709082 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d539898c-2ba0-446d-b0ef-c9843d745a70 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:47.710354 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 224/456] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 11:59:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:59:47.715727 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa8b2f7-9bfd-4304-8607-d9a65ea96226 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:47.716458 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa8b2f7-9bfd-4304-8607-d9a65ea96226 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:47.716873 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa8b2f7-9bfd-4304-8607-d9a65ea96226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:47.748515 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-ffa8b2f7-9bfd-4304-8607-d9a65ea96226 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:59:47.749714 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-ffa8b2f7-9bfd-4304-8607-d9a65ea96226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:47.853521 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 233/457] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 11:59:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 11:59:47.929017 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48049c3-93ec-4940-8fdc-67ace3dff9d9 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:47.929292 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48049c3-93ec-4940-8fdc-67ace3dff9d9 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:47.929468 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48049c3-93ec-4940-8fdc-67ace3dff9d9 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:47.931212 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 225/458] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 11:59:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:59:47.936660 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ef59ef-5195-4d12-a209-083eb40e00c7 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:47.937115 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ef59ef-5195-4d12-a209-083eb40e00c7 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:47.937526 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ef59ef-5195-4d12-a209-083eb40e00c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:47.965150 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-36ef59ef-5195-4d12-a209-083eb40e00c7 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:59:47.966582 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-36ef59ef-5195-4d12-a209-083eb40e00c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:48.064375 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 234/459] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 11:59:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:59:48.225068 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48049c3-93ec-4940-8fdc-67ace3dff9d9 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:48.261459 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:48.345624 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48049c3-93ec-4940-8fdc-67ace3dff9d9 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:48.387967 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2077bbcf-cf9d-472a-acd0-18816e900cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:48.388986 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2077bbcf-cf9d-472a-acd0-18816e900cd0 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:48.389463 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2077bbcf-cf9d-472a-acd0-18816e900cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:48.389927 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2077bbcf-cf9d-472a-acd0-18816e900cd0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:48.402609 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2077bbcf-cf9d-472a-acd0-18816e900cd0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:48.404461 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2077bbcf-cf9d-472a-acd0-18816e900cd0 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:48.424632 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-2077bbcf-cf9d-472a-acd0-18816e900cd0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:59:48.424632 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:59:48.424632 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:59:48.424632 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:59:48.424632 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:59:48.424632 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:59:48.424632 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:59:48.424632 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:59:48.424632 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:59:48.424632 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:59:48.424632 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:59:48.424632 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:59:48.424632 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:59:48.424632 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:59:48.424632 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:59:48.424632 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:59:48.425709 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:59:48.425709 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:59:48.425709 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:59:48.425709 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:59:48.425709 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:59:48.425709 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:59:48.425709 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:59:48.425709 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 11:59:48.425709 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:59:48.425709 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:59:48.425709 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:59:48.425709 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:59:48.425709 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:59:48.425709 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:59:48.425709 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:59:48.425709 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:59:48.425709 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:59:48.427166 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:59:48.427166 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:59:48.427166 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 11:59:48.429031 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 226/460] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 11:59:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:59:48.435135 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ef59ef-5195-4d12-a209-083eb40e00c7 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:48.470652 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4bc2e8-4662-412e-95d9-4a5b3ea7645f None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:48.471127 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4bc2e8-4662-412e-95d9-4a5b3ea7645f None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:48.471408 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4bc2e8-4662-412e-95d9-4a5b3ea7645f None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:48.473057 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 227/461] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 11:59:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:59:48.482867 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446007b1-62ae-4dd4-93eb-8aa6f4844081 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:48.483153 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446007b1-62ae-4dd4-93eb-8aa6f4844081 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:48.483372 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446007b1-62ae-4dd4-93eb-8aa6f4844081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:48.503295 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-446007b1-62ae-4dd4-93eb-8aa6f4844081 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:59:48.505080 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-446007b1-62ae-4dd4-93eb-8aa6f4844081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:48.513502 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:48.572796 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ef59ef-5195-4d12-a209-083eb40e00c7 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:48.580348 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 228/462] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 11:59:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 11:59:48.581430 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446007b1-62ae-4dd4-93eb-8aa6f4844081 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:48.595742 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:48.617689 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614991c2-4845-4d1b-adfd-89c86c4cdcf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:48.618496 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614991c2-4845-4d1b-adfd-89c86c4cdcf0 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:48.618980 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614991c2-4845-4d1b-adfd-89c86c4cdcf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:48.619587 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614991c2-4845-4d1b-adfd-89c86c4cdcf0 None admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:48.620692 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614991c2-4845-4d1b-adfd-89c86c4cdcf0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:59:48.621143 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614991c2-4845-4d1b-adfd-89c86c4cdcf0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:48.622918 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614991c2-4845-4d1b-adfd-89c86c4cdcf0 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:48.651401 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 235/463] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 9 11:59:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:48.652919 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb1b084-9d36-4a64-9765-5c04b2ce2744 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:48.653304 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb1b084-9d36-4a64-9765-5c04b2ce2744 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:48.653674 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb1b084-9d36-4a64-9765-5c04b2ce2744 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:48.655565 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 236/464] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 11:59:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:59:48.656406 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb1b084-9d36-4a64-9765-5c04b2ce2744 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:48.712164 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446007b1-62ae-4dd4-93eb-8aa6f4844081 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:48.750225 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2c4b1a-7601-484b-af7e-ae59f17975fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:48.751121 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2c4b1a-7601-484b-af7e-ae59f17975fa None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:48.751571 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2c4b1a-7601-484b-af7e-ae59f17975fa None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:48.752051 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2c4b1a-7601-484b-af7e-ae59f17975fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:48.764728 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2c4b1a-7601-484b-af7e-ae59f17975fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:48.766710 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2c4b1a-7601-484b-af7e-ae59f17975fa None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:48.767314 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb1b084-9d36-4a64-9765-5c04b2ce2744 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:48.769846 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:48.786782 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-5d2c4b1a-7601-484b-af7e-ae59f17975fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:59:48.786782 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:59:48.786782 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:59:48.786782 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:59:48.786782 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:59:48.786782 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:59:48.786782 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:59:48.786782 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:59:48.786782 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:59:48.786782 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:59:48.786782 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:59:48.786782 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:59:48.786782 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:59:48.786782 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:59:48.786782 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:59:48.786782 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:59:48.787962 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:59:48.787962 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:59:48.787962 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:59:48.787962 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:59:48.787962 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:59:48.787962 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:59:48.787962 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:59:48.787962 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 11:59:48.787962 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:59:48.787962 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:59:48.787962 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:59:48.787962 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:59:48.787962 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:59:48.787962 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:59:48.787962 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:59:48.787962 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:59:48.787962 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:59:48.788974 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:59:48.788974 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:59:48.788974 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 11:59:48.790624 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 229/465] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 11:59:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:59:48.792048 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a87c03-3eff-4840-99ec-096e0ada96b2 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:48.792536 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a87c03-3eff-4840-99ec-096e0ada96b2 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:48.792952 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a87c03-3eff-4840-99ec-096e0ada96b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:48.810629 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812708d5-bf7f-40e9-bec2-e23bec90a52f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:48.811267 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812708d5-bf7f-40e9-bec2-e23bec90a52f None admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:48.811544 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812708d5-bf7f-40e9-bec2-e23bec90a52f None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:48.811812 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812708d5-bf7f-40e9-bec2-e23bec90a52f None admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:48.812452 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812708d5-bf7f-40e9-bec2-e23bec90a52f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:48.813838 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-f2a87c03-3eff-4840-99ec-096e0ada96b2 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:59:48.813955 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812708d5-bf7f-40e9-bec2-e23bec90a52f None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:48.815569 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-f2a87c03-3eff-4840-99ec-096e0ada96b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:48.840831 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 237/466] 10.222.0.49 () {60 vars in 1132 bytes} [Fri Sep 9 11:59:48 2022] POST /identity/v3/users => generated 246 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 11:59:48.848409 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812708d5-bf7f-40e9-bec2-e23bec90a52f None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:48.883766 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:48.896669 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 230/467] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 11:59:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:59:48.903667 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a87c03-3eff-4840-99ec-096e0ada96b2 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:48.973138 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812708d5-bf7f-40e9-bec2-e23bec90a52f None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:49.015304 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4c0acb-b718-4eac-8dd6-5459f272284c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:49.016174 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4c0acb-b718-4eac-8dd6-5459f272284c None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:49.016603 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4c0acb-b718-4eac-8dd6-5459f272284c None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:49.017023 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4c0acb-b718-4eac-8dd6-5459f272284c None admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:49.017817 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4c0acb-b718-4eac-8dd6-5459f272284c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:59:49.018258 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4c0acb-b718-4eac-8dd6-5459f272284c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:49.019839 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4c0acb-b718-4eac-8dd6-5459f272284c None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:49.035051 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 238/468] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 9 11:59:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:49.042555 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4c0acb-b718-4eac-8dd6-5459f272284c None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:49.069188 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:49.108012 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a87c03-3eff-4840-99ec-096e0ada96b2 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:49.157709 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f280bd26-3d3b-4c96-a0ab-f1ee483218d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:49.158685 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f280bd26-3d3b-4c96-a0ab-f1ee483218d6 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:49.159130 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f280bd26-3d3b-4c96-a0ab-f1ee483218d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:49.159540 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f280bd26-3d3b-4c96-a0ab-f1ee483218d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:49.167375 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4c0acb-b718-4eac-8dd6-5459f272284c None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:49.171444 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f280bd26-3d3b-4c96-a0ab-f1ee483218d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:49.173141 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f280bd26-3d3b-4c96-a0ab-f1ee483218d6 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:49.190174 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-f280bd26-3d3b-4c96-a0ab-f1ee483218d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:59:49.190174 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:59:49.190174 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:59:49.190174 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:59:49.190174 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:59:49.190174 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:59:49.190174 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:59:49.190174 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:59:49.190174 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:59:49.190174 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:59:49.190174 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:59:49.190174 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:59:49.190174 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:59:49.190174 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:59:49.190174 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:59:49.190174 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:59:49.191328 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:59:49.191328 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:59:49.191328 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:59:49.191328 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:59:49.191328 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:59:49.191328 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:59:49.191328 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:59:49.191328 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 11:59:49.191328 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:59:49.191328 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:59:49.191328 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:59:49.191328 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:59:49.191328 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:59:49.191328 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:59:49.191328 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:59:49.191328 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:59:49.191328 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:59:49.193179 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:59:49.193179 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:59:49.193179 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 11:59:49.194697 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 231/469] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 11:59:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:59:49.200619 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f280bd26-3d3b-4c96-a0ab-f1ee483218d6 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:49.213207 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a3770e-9fc4-4166-a9bb-056383ae5579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:49.213827 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a3770e-9fc4-4166-a9bb-056383ae5579 None admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:49.214125 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a3770e-9fc4-4166-a9bb-056383ae5579 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:49.214581 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a3770e-9fc4-4166-a9bb-056383ae5579 None admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:49.215062 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a3770e-9fc4-4166-a9bb-056383ae5579 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:49.216507 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a3770e-9fc4-4166-a9bb-056383ae5579 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:49.248477 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 239/470] 10.222.0.49 () {60 vars in 1132 bytes} [Fri Sep 9 11:59:49 2022] POST /identity/v3/users => generated 248 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 11:59:49.254601 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:49.293591 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a3770e-9fc4-4166-a9bb-056383ae5579 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:49.334803 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f280bd26-3d3b-4c96-a0ab-f1ee483218d6 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:49.373467 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8fe180-a5b7-420e-862e-07175a433bf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:49.374093 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8fe180-a5b7-420e-862e-07175a433bf6 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:49.374372 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8fe180-a5b7-420e-862e-07175a433bf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:49.375035 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8fe180-a5b7-420e-862e-07175a433bf6 None admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:49.375330 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8fe180-a5b7-420e-862e-07175a433bf6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:59:49.375635 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8fe180-a5b7-420e-862e-07175a433bf6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:49.377055 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8fe180-a5b7-420e-862e-07175a433bf6 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:49.391675 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 232/471] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 9 11:59:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:59:49.398596 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8fe180-a5b7-420e-862e-07175a433bf6 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:49.436785 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:49.468598 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a3770e-9fc4-4166-a9bb-056383ae5579 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:49.505756 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f551a207-6be8-4c2e-b1e4-5905a576af2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:49.506388 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f551a207-6be8-4c2e-b1e4-5905a576af2d None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:49.506686 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f551a207-6be8-4c2e-b1e4-5905a576af2d None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:49.506992 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f551a207-6be8-4c2e-b1e4-5905a576af2d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:49.522211 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f551a207-6be8-4c2e-b1e4-5905a576af2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:49.524193 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f551a207-6be8-4c2e-b1e4-5905a576af2d None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:49.534865 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8fe180-a5b7-420e-862e-07175a433bf6 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:49.537327 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:49.539638 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-f551a207-6be8-4c2e-b1e4-5905a576af2d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:59:49.539638 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:59:49.539638 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:59:49.539638 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:59:49.539638 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:59:49.539638 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:59:49.539638 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:59:49.539638 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:59:49.539638 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:59:49.539638 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:59:49.539638 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:59:49.539638 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:59:49.539638 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:59:49.539638 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:59:49.539638 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:59:49.539638 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:59:49.540568 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:59:49.540568 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:59:49.540568 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:59:49.540568 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:59:49.540568 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:59:49.540568 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:59:49.540568 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:59:49.540568 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 11:59:49.540568 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:59:49.540568 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:59:49.540568 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:59:49.540568 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:59:49.540568 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:59:49.540568 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:59:49.540568 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:59:49.540568 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:59:49.540568 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:59:49.541899 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:59:49.541899 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:59:49.541899 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 11:59:49.541899 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 240/472] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 11:59:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:59:49.543896 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718e991c-4385-441f-af29-a3dd80b42ccd None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:49.543896 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718e991c-4385-441f-af29-a3dd80b42ccd None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:49.547110 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718e991c-4385-441f-af29-a3dd80b42ccd None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:49.548469 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 241/473] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 11:59:49 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:59:49.549200 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718e991c-4385-441f-af29-a3dd80b42ccd None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:49.588269 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af4aef7-f91e-4715-8a82-ee6ec8e7a065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:49.588912 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af4aef7-f91e-4715-8a82-ee6ec8e7a065 None admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:49.589192 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af4aef7-f91e-4715-8a82-ee6ec8e7a065 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:49.589457 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af4aef7-f91e-4715-8a82-ee6ec8e7a065 None admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:49.590104 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af4aef7-f91e-4715-8a82-ee6ec8e7a065 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:49.591586 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af4aef7-f91e-4715-8a82-ee6ec8e7a065 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:49.616214 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 233/474] 10.222.0.49 () {60 vars in 1132 bytes} [Fri Sep 9 11:59:49 2022] POST /identity/v3/users => generated 248 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:59:49.619092 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e77ef8-c2d2-4266-8b89-73f2113b7a28 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:49.619436 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e77ef8-c2d2-4266-8b89-73f2113b7a28 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:49.619753 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e77ef8-c2d2-4266-8b89-73f2113b7a28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:49.647228 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-31e77ef8-c2d2-4266-8b89-73f2113b7a28 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:59:49.648616 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-31e77ef8-c2d2-4266-8b89-73f2113b7a28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:49.682564 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718e991c-4385-441f-af29-a3dd80b42ccd None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:49.716149 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:49.727432 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d2eb4b-2f9a-476f-950b-ab84efbb700e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:49.728463 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d2eb4b-2f9a-476f-950b-ab84efbb700e None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:49.729077 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d2eb4b-2f9a-476f-950b-ab84efbb700e None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:49.729579 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d2eb4b-2f9a-476f-950b-ab84efbb700e None admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:49.730431 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d2eb4b-2f9a-476f-950b-ab84efbb700e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:59:49.730938 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d2eb4b-2f9a-476f-950b-ab84efbb700e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:49.732973 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d2eb4b-2f9a-476f-950b-ab84efbb700e None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:49.734475 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 234/475] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 11:59:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:59:49.748544 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 242/476] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 9 11:59:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:49.755191 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e77ef8-c2d2-4266-8b89-73f2113b7a28 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:49.856331 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:49.869337 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e77ef8-c2d2-4266-8b89-73f2113b7a28 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:49.907159 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a15bbd4-6e42-45c4-9d7d-747c4ffa8e7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:49.908417 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a15bbd4-6e42-45c4-9d7d-747c4ffa8e7a None admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:49.908878 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a15bbd4-6e42-45c4-9d7d-747c4ffa8e7a None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:49.909325 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a15bbd4-6e42-45c4-9d7d-747c4ffa8e7a None admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:49.910149 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a15bbd4-6e42-45c4-9d7d-747c4ffa8e7a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:49.911870 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a15bbd4-6e42-45c4-9d7d-747c4ffa8e7a None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:49.936257 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 235/477] 10.222.0.49 () {60 vars in 1132 bytes} [Fri Sep 9 11:59:49 2022] POST /identity/v3/users => generated 249 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:59:50.105925 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d2eb4b-2f9a-476f-950b-ab84efbb700e None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:50.223792 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d2eb4b-2f9a-476f-950b-ab84efbb700e None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:50.266762 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3301b227-28ef-4e08-b290-5a9f261b8120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:50.267843 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3301b227-28ef-4e08-b290-5a9f261b8120 None admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:50.268325 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3301b227-28ef-4e08-b290-5a9f261b8120 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:50.268847 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3301b227-28ef-4e08-b290-5a9f261b8120 None admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:50.269812 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3301b227-28ef-4e08-b290-5a9f261b8120 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:50.271433 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3301b227-28ef-4e08-b290-5a9f261b8120 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:50.281027 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.sql.core [None req-3301b227-28ef-4e08-b290-5a9f261b8120 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 09 11:59:50.281027 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 09 11:59:50.281027 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [parameters: {'id': '24139ed6f9b745188d59d9e4cbec2b3f', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 09 11:59:50.281027 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81591) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 09 11:59:50.298847 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-3301b227-28ef-4e08-b290-5a9f261b8120 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 09 11:59:50.298847 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:59:50.298847 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 09 11:59:50.298847 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 09 11:59:50.298847 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 09 11:59:50.298847 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 09 11:59:50.298847 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 09 11:59:50.298847 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application result = self._query(query) Sep 09 11:59:50.298847 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 09 11:59:50.298847 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application conn.query(q) Sep 09 11:59:50.298847 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 09 11:59:50.298847 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 09 11:59:50.298847 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 09 11:59:50.298847 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application result.read() Sep 09 11:59:50.298847 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 09 11:59:50.299796 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 09 11:59:50.299796 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 09 11:59:50.299796 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 09 11:59:50.299796 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 09 11:59:50.299796 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 09 11:59:50.299796 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 09 11:59:50.299796 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 09 11:59:50.299796 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 09 11:59:50.299796 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 11:59:50.299796 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 09 11:59:50.299796 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 11:59:50.299796 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:59:50.299796 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 09 11:59:50.299796 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 09 11:59:50.299796 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 09 11:59:50.299796 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return ref.to_dict() Sep 09 11:59:50.299796 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 09 11:59:50.299796 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application next(self.gen) Sep 09 11:59:50.300759 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 09 11:59:50.300759 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application yield resource Sep 09 11:59:50.300759 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 09 11:59:50.300759 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application next(self.gen) Sep 09 11:59:50.300759 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 09 11:59:50.300759 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application self.session.rollback() Sep 09 11:59:50.300759 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 09 11:59:50.300759 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application self.force_reraise() Sep 09 11:59:50.300759 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 09 11:59:50.300759 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise self.value Sep 09 11:59:50.300759 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 09 11:59:50.300759 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 09 11:59:50.300759 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 09 11:59:50.300759 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application session.commit() Sep 09 11:59:50.300759 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 09 11:59:50.300759 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 09 11:59:50.300759 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 09 11:59:50.302252 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application self._prepare_impl() Sep 09 11:59:50.302252 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 09 11:59:50.302252 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application self.session.flush() Sep 09 11:59:50.302252 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 09 11:59:50.302252 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application self._flush(objects) Sep 09 11:59:50.302252 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 09 11:59:50.302252 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 09 11:59:50.302252 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 09 11:59:50.302252 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application compat.raise_( Sep 09 11:59:50.302252 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 09 11:59:50.302252 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception Sep 09 11:59:50.302252 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 09 11:59:50.302252 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application flush_context.execute() Sep 09 11:59:50.302252 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 09 11:59:50.302252 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rec.execute(self) Sep 09 11:59:50.302252 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 09 11:59:50.302252 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 09 11:59:50.303374 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 09 11:59:50.303374 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 09 11:59:50.303374 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 09 11:59:50.303374 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 09 11:59:50.303374 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 09 11:59:50.303374 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 09 11:59:50.303374 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 09 11:59:50.303374 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 09 11:59:50.303374 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 09 11:59:50.303374 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 09 11:59:50.303374 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 09 11:59:50.303374 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 09 11:59:50.303374 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 09 11:59:50.303374 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 09 11:59:50.303374 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 09 11:59:50.303374 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception Sep 09 11:59:50.304201 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 09 11:59:50.304201 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 09 11:59:50.304201 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 09 11:59:50.304201 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 09 11:59:50.304201 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 09 11:59:50.304201 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application result = self._query(query) Sep 09 11:59:50.304201 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 09 11:59:50.304201 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application conn.query(q) Sep 09 11:59:50.304201 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 09 11:59:50.304201 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 09 11:59:50.304201 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 09 11:59:50.304201 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application result.read() Sep 09 11:59:50.304201 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 09 11:59:50.304201 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 09 11:59:50.304201 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 09 11:59:50.304201 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 09 11:59:50.304991 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 09 11:59:50.304991 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 09 11:59:50.304991 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 09 11:59:50.304991 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 09 11:59:50.304991 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 09 11:59:50.304991 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 09 11:59:50.304991 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [parameters: {'id': '24139ed6f9b745188d59d9e4cbec2b3f', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 09 11:59:50.304991 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 09 11:59:50.304991 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 11:59:50.304991 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 11:59:50.304991 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 11:59:50.304991 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:59:50.304991 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:59:50.304991 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:59:50.304991 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:59:50.304991 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:59:50.305799 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:59:50.305799 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:59:50.305799 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:59:50.305799 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:59:50.305799 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:59:50.305799 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:59:50.305799 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 09 11:59:50.305799 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 09 11:59:50.305799 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:59:50.305799 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:59:50.305799 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 09 11:59:50.305799 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 09 11:59:50.305799 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 09 11:59:50.305799 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 09 11:59:50.305799 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 09 11:59:50.305799 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 11:59:50.309951 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 243/478] 10.222.0.49 () {60 vars in 1131 bytes} [Fri Sep 9 11:59:50 2022] POST /identity/v3/roles => generated 144 bytes in 204 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:59:50.316234 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a15bbd4-6e42-45c4-9d7d-747c4ffa8e7a None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:50.426125 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a15bbd4-6e42-45c4-9d7d-747c4ffa8e7a None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:50.463915 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65c6d0c-693b-4688-adc9-a0fba3f94551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:50.464912 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65c6d0c-693b-4688-adc9-a0fba3f94551 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:50.465421 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65c6d0c-693b-4688-adc9-a0fba3f94551 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:50.465920 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65c6d0c-693b-4688-adc9-a0fba3f94551 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:50.478530 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65c6d0c-693b-4688-adc9-a0fba3f94551 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:59:50.484600 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-e65c6d0c-693b-4688-adc9-a0fba3f94551 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 11:59:50.484600 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:59:50.484600 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:59:50.484600 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:59:50.484600 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:59:50.484600 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:59:50.484600 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:59:50.484600 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:59:50.484600 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:59:50.484600 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:59:50.484600 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:59:50.484600 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:59:50.484600 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:59:50.484600 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:59:50.484600 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:59:50.484600 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:59:50.485572 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:59:50.485572 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:59:50.485572 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:59:50.485572 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:59:50.485572 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:59:50.485572 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:59:50.485572 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:59:50.485572 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:59:50.485572 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:59:50.485572 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:59:50.485572 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:59:50.485572 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 11:59:50.485572 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:59:50.485572 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:59:50.485572 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:59:50.485572 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:59:50.486385 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:59:50.486385 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:59:50.486385 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:59:50.486385 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:59:50.486385 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:59:50.486385 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:59:50.486385 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:59:50.486385 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:59:50.486385 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:59:50.486385 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:59:50.486385 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 11:59:50.486385 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 11:59:50.489632 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 236/479] 10.222.0.49 () {56 vars in 1114 bytes} [Fri Sep 9 11:59:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:59:50.495974 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3301b227-28ef-4e08-b290-5a9f261b8120 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:50.558991 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:50.625700 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3301b227-28ef-4e08-b290-5a9f261b8120 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:50.677205 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d551d3-dc30-4e6a-b6da-2483a5699e65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:50.678262 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d551d3-dc30-4e6a-b6da-2483a5699e65 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:50.678742 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d551d3-dc30-4e6a-b6da-2483a5699e65 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:50.679234 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d551d3-dc30-4e6a-b6da-2483a5699e65 None admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:50.680106 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d551d3-dc30-4e6a-b6da-2483a5699e65 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:59:50.680621 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d551d3-dc30-4e6a-b6da-2483a5699e65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:50.682198 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d551d3-dc30-4e6a-b6da-2483a5699e65 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:50.695565 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 244/480] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 9 11:59:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:51.745900 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cdd85d-9904-46be-a41b-ff3e630a3f32 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:51.746409 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cdd85d-9904-46be-a41b-ff3e630a3f32 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:51.746517 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cdd85d-9904-46be-a41b-ff3e630a3f32 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:51.748037 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 237/481] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 11:59:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:59:51.753524 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dce313-5d7e-449a-852d-b3e3613dcc1e None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:51.753937 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dce313-5d7e-449a-852d-b3e3613dcc1e None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:51.754109 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dce313-5d7e-449a-852d-b3e3613dcc1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:51.773511 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-24dce313-5d7e-449a-852d-b3e3613dcc1e None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:59:51.779132 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-24dce313-5d7e-449a-852d-b3e3613dcc1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:51.871858 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 245/482] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 11:59:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 11:59:52.308363 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cdd85d-9904-46be-a41b-ff3e630a3f32 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:52.312531 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:52.320822 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbd2913-17cd-4c0a-8a3f-96b787c13043 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:52.321123 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbd2913-17cd-4c0a-8a3f-96b787c13043 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:52.321382 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbd2913-17cd-4c0a-8a3f-96b787c13043 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:52.326867 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 246/483] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 11:59:52 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:59:52.328734 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5e751d-463a-4d09-b6f3-33ad7ec0b709 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:52.329024 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5e751d-463a-4d09-b6f3-33ad7ec0b709 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:52.329288 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5e751d-463a-4d09-b6f3-33ad7ec0b709 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:52.358619 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-cb5e751d-463a-4d09-b6f3-33ad7ec0b709 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:59:52.360407 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-cb5e751d-463a-4d09-b6f3-33ad7ec0b709 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:52.411290 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:52.448828 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 247/484] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 11:59:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:59:52.450445 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e5f295-6b18-4817-81bc-d06d6e659022 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:52.450955 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e5f295-6b18-4817-81bc-d06d6e659022 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:52.451467 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e5f295-6b18-4817-81bc-d06d6e659022 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:52.452901 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 248/485] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 11:59:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:59:52.456525 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cdd85d-9904-46be-a41b-ff3e630a3f32 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:52.472738 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a7239f-9b35-4e6c-be9f-0ef21a46afb5 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:52.473431 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a7239f-9b35-4e6c-be9f-0ef21a46afb5 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:52.473905 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a7239f-9b35-4e6c-be9f-0ef21a46afb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:52.501222 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-a1a7239f-9b35-4e6c-be9f-0ef21a46afb5 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:59:52.504196 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-a1a7239f-9b35-4e6c-be9f-0ef21a46afb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:52.525056 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0743bd-4cb4-4dd1-b9f0-e6c3a4093eec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:52.526070 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0743bd-4cb4-4dd1-b9f0-e6c3a4093eec None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:52.526548 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0743bd-4cb4-4dd1-b9f0-e6c3a4093eec None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:52.527057 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0743bd-4cb4-4dd1-b9f0-e6c3a4093eec None admin] PATH_INFO: `/v3/roles/service` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:52.540523 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0743bd-4cb4-4dd1-b9f0-e6c3a4093eec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:59:52.551455 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-9b0743bd-4cb4-4dd1-b9f0-e6c3a4093eec None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 11:59:52.551455 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:59:52.551455 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:59:52.551455 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:59:52.551455 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:59:52.551455 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:59:52.551455 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:59:52.551455 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:59:52.551455 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:59:52.551455 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:59:52.551455 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:59:52.551455 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:59:52.551455 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:59:52.551455 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:59:52.551455 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:59:52.551455 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:59:52.553039 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:59:52.553039 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:59:52.553039 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:59:52.553039 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:59:52.553039 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:59:52.553039 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:59:52.553039 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:59:52.553039 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:59:52.553039 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:59:52.553039 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:59:52.553039 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:59:52.553039 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 11:59:52.553039 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:59:52.553039 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:59:52.553039 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:59:52.553039 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:59:52.554195 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:59:52.554195 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:59:52.554195 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:59:52.554195 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:59:52.554195 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:59:52.554195 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:59:52.554195 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:59:52.554195 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:59:52.554195 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:59:52.554195 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:59:52.554195 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 11:59:52.554195 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 11:59:52.560499 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 238/486] 10.222.0.49 () {56 vars in 1102 bytes} [Fri Sep 9 11:59:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:59:52.568430 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0743bd-4cb4-4dd1-b9f0-e6c3a4093eec None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:52.684010 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 249/487] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 11:59:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:59:52.706116 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:52.749876 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0743bd-4cb4-4dd1-b9f0-e6c3a4093eec None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:52.787879 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b826f122-6eda-4408-a30f-0126bc82bf55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:52.788643 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b826f122-6eda-4408-a30f-0126bc82bf55 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:52.788919 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b826f122-6eda-4408-a30f-0126bc82bf55 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:52.789191 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b826f122-6eda-4408-a30f-0126bc82bf55 None admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:52.789942 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b826f122-6eda-4408-a30f-0126bc82bf55 None admin] RBAC: Adding query filter params (name=service) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:59:52.790248 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b826f122-6eda-4408-a30f-0126bc82bf55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:52.791738 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b826f122-6eda-4408-a30f-0126bc82bf55 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:52.802227 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 239/488] 10.222.0.49 () {56 vars in 1103 bytes} [Fri Sep 9 11:59:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:59:52.808234 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a7239f-9b35-4e6c-be9f-0ef21a46afb5 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:52.828948 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b826f122-6eda-4408-a30f-0126bc82bf55 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:52.971518 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b826f122-6eda-4408-a30f-0126bc82bf55 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:52.982556 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a7239f-9b35-4e6c-be9f-0ef21a46afb5 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:52.985594 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:53.004025 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:53.022115 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d2514a-a445-43dc-909d-2019d04a5134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:53.023032 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d2514a-a445-43dc-909d-2019d04a5134 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:53.023307 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d2514a-a445-43dc-909d-2019d04a5134 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:53.023636 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d2514a-a445-43dc-909d-2019d04a5134 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:53.029463 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea58fa31-ea65-4612-80aa-19f0559074b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:53.030216 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea58fa31-ea65-4612-80aa-19f0559074b4 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:53.030591 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea58fa31-ea65-4612-80aa-19f0559074b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:53.031010 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea58fa31-ea65-4612-80aa-19f0559074b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:53.037249 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d2514a-a445-43dc-909d-2019d04a5134 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:59:53.042296 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea58fa31-ea65-4612-80aa-19f0559074b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:53.045042 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-75d2514a-a445-43dc-909d-2019d04a5134 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 11:59:53.045042 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:59:53.045042 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:59:53.045042 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:59:53.045042 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:59:53.045042 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:59:53.045042 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:59:53.045042 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:59:53.045042 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:59:53.045042 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:59:53.045042 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:59:53.045042 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:59:53.045042 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:59:53.045042 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:59:53.045042 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:59:53.045042 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:59:53.045916 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:59:53.045916 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:59:53.045916 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:59:53.045916 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:59:53.045916 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:59:53.045916 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:59:53.045916 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:59:53.045916 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:59:53.045916 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:59:53.045916 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:59:53.045916 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:59:53.045916 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 11:59:53.045916 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:59:53.045916 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:59:53.045916 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:59:53.045916 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:59:53.047554 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:59:53.047554 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:59:53.047554 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:59:53.047554 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:59:53.047554 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:59:53.047554 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:59:53.047554 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:59:53.047554 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:59:53.047554 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:59:53.047554 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:59:53.047554 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 11:59:53.047554 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 11:59:53.048601 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea58fa31-ea65-4612-80aa-19f0559074b4 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:53.048708 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 240/489] 10.222.0.49 () {56 vars in 1102 bytes} [Fri Sep 9 11:59:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:59:53.048708 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0fc3e6-1dd7-4438-9637-41206398d00c None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:53.048708 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0fc3e6-1dd7-4438-9637-41206398d00c None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:53.048708 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0fc3e6-1dd7-4438-9637-41206398d00c None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:53.050477 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 241/490] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 11:59:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:59:53.056044 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0fc3e6-1dd7-4438-9637-41206398d00c None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:53.070457 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-ea58fa31-ea65-4612-80aa-19f0559074b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:59:53.070457 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:59:53.070457 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:59:53.070457 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:59:53.070457 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:59:53.070457 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:59:53.070457 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:59:53.070457 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:59:53.070457 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:59:53.070457 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:59:53.070457 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:59:53.070457 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:59:53.070457 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:59:53.070457 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:59:53.070457 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:59:53.070457 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:59:53.071671 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:59:53.071671 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:59:53.071671 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:59:53.071671 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:59:53.071671 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:59:53.071671 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:59:53.071671 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:59:53.071671 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 11:59:53.071671 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:59:53.071671 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:59:53.071671 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:59:53.071671 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:59:53.071671 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:59:53.071671 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:59:53.071671 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:59:53.071671 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:59:53.071671 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:59:53.072572 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:59:53.072572 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:59:53.072572 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 11:59:53.072572 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 250/491] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 11:59:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:59:53.076712 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea58fa31-ea65-4612-80aa-19f0559074b4 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:53.172019 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0fc3e6-1dd7-4438-9637-41206398d00c None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:53.184449 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea58fa31-ea65-4612-80aa-19f0559074b4 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:53.209205 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53ba92f-ba9b-408e-b45e-ba3bd019d0d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:53.209807 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53ba92f-ba9b-408e-b45e-ba3bd019d0d9 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:53.210091 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53ba92f-ba9b-408e-b45e-ba3bd019d0d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:53.210362 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53ba92f-ba9b-408e-b45e-ba3bd019d0d9 None admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:53.211079 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53ba92f-ba9b-408e-b45e-ba3bd019d0d9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:59:53.211400 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53ba92f-ba9b-408e-b45e-ba3bd019d0d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:53.212757 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53ba92f-ba9b-408e-b45e-ba3bd019d0d9 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:53.220679 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 242/492] 10.222.0.49 () {56 vars in 1103 bytes} [Fri Sep 9 11:59:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:53.221864 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5379c01d-9886-4e84-8405-81077216d2b9 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:53.222148 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5379c01d-9886-4e84-8405-81077216d2b9 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:53.222418 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5379c01d-9886-4e84-8405-81077216d2b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:53.227962 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60d6ddd-7225-4f81-bf65-dd3f598d26da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:53.228706 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60d6ddd-7225-4f81-bf65-dd3f598d26da None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:53.228985 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60d6ddd-7225-4f81-bf65-dd3f598d26da None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:53.229302 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60d6ddd-7225-4f81-bf65-dd3f598d26da None admin] PATH_INFO: `/v3/roles/service` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:53.241506 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-5379c01d-9886-4e84-8405-81077216d2b9 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:59:53.242587 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60d6ddd-7225-4f81-bf65-dd3f598d26da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:59:53.242711 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-5379c01d-9886-4e84-8405-81077216d2b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:53.248265 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-e60d6ddd-7225-4f81-bf65-dd3f598d26da None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 11:59:53.248265 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:59:53.248265 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:59:53.248265 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:59:53.248265 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:59:53.248265 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:59:53.248265 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:59:53.248265 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:59:53.248265 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:59:53.248265 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:59:53.248265 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:59:53.248265 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:59:53.248265 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:59:53.248265 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:59:53.248265 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:59:53.248265 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:59:53.249163 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:59:53.249163 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:59:53.249163 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:59:53.249163 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:59:53.249163 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:59:53.249163 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:59:53.249163 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:59:53.249163 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:59:53.249163 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:59:53.249163 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:59:53.249163 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:59:53.249163 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 11:59:53.249163 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:59:53.249163 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:59:53.249163 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:59:53.249163 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:59:53.252183 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:59:53.252183 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:59:53.252183 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:59:53.252183 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:59:53.252183 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:59:53.252183 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:59:53.252183 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:59:53.252183 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:59:53.252183 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:59:53.252183 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:59:53.252183 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 11:59:53.252183 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 11:59:53.252183 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 251/493] 10.222.0.49 () {56 vars in 1102 bytes} [Fri Sep 9 11:59:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:59:53.252183 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60d6ddd-7225-4f81-bf65-dd3f598d26da None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:53.273961 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:53.295338 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:53.323842 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 243/494] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 11:59:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:59:53.324692 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5379c01d-9886-4e84-8405-81077216d2b9 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:53.386583 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60d6ddd-7225-4f81-bf65-dd3f598d26da None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:53.429966 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee8c816-748e-438e-8d68-a1e40364e047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:53.430848 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee8c816-748e-438e-8d68-a1e40364e047 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:53.431340 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee8c816-748e-438e-8d68-a1e40364e047 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:53.431973 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee8c816-748e-438e-8d68-a1e40364e047 None admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:53.433122 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee8c816-748e-438e-8d68-a1e40364e047 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:59:53.433617 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee8c816-748e-438e-8d68-a1e40364e047 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:53.435189 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee8c816-748e-438e-8d68-a1e40364e047 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:53.438649 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5379c01d-9886-4e84-8405-81077216d2b9 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:53.448311 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 252/495] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 9 11:59:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:53.449316 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee8c816-748e-438e-8d68-a1e40364e047 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:53.479045 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5f131a-9d43-42e3-be45-7ba97ec5b428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:53.479678 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5f131a-9d43-42e3-be45-7ba97ec5b428 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:53.479959 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5f131a-9d43-42e3-be45-7ba97ec5b428 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:53.480306 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5f131a-9d43-42e3-be45-7ba97ec5b428 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:53.491068 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5f131a-9d43-42e3-be45-7ba97ec5b428 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:53.492520 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5f131a-9d43-42e3-be45-7ba97ec5b428 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:53.507135 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-8c5f131a-9d43-42e3-be45-7ba97ec5b428 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:59:53.507135 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:59:53.507135 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:59:53.507135 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:59:53.507135 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:59:53.507135 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:59:53.507135 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:59:53.507135 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:59:53.507135 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:59:53.507135 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:59:53.507135 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:59:53.507135 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:59:53.507135 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:59:53.507135 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:59:53.507135 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:59:53.507135 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:59:53.508063 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:59:53.508063 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:59:53.508063 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:59:53.508063 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:59:53.508063 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:59:53.508063 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:59:53.508063 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:59:53.508063 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 11:59:53.508063 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:59:53.508063 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:59:53.508063 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:59:53.508063 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:59:53.508063 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:59:53.508063 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:59:53.508063 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:59:53.508063 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:59:53.508063 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:59:53.509522 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:59:53.509522 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:59:53.509522 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 11:59:53.509522 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 244/496] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 11:59:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:59:53.509736 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d20274-4a20-4f16-af33-3872cb77350e None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:53.510039 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d20274-4a20-4f16-af33-3872cb77350e None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:53.510228 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d20274-4a20-4f16-af33-3872cb77350e None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:53.511720 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 245/497] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 11:59:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:59:53.512420 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d20274-4a20-4f16-af33-3872cb77350e None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:53.539054 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:53.545123 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:53.566145 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee8c816-748e-438e-8d68-a1e40364e047 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:53.603705 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785c13c1-79e5-4b13-84b0-234f70fbac32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:53.604589 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785c13c1-79e5-4b13-84b0-234f70fbac32 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:53.605059 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785c13c1-79e5-4b13-84b0-234f70fbac32 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:53.605335 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785c13c1-79e5-4b13-84b0-234f70fbac32 None admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:53.606003 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785c13c1-79e5-4b13-84b0-234f70fbac32 None admin] RBAC: Adding query filter params (name=service) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:59:53.606308 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785c13c1-79e5-4b13-84b0-234f70fbac32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:53.607720 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785c13c1-79e5-4b13-84b0-234f70fbac32 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:53.616554 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 253/498] 10.222.0.49 () {56 vars in 1103 bytes} [Fri Sep 9 11:59:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:53.617354 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785c13c1-79e5-4b13-84b0-234f70fbac32 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:53.622862 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d20274-4a20-4f16-af33-3872cb77350e None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:53.662316 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2443d1f8-f62f-49bb-8b0e-e05ee79d30c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:53.662979 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2443d1f8-f62f-49bb-8b0e-e05ee79d30c9 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:53.663262 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2443d1f8-f62f-49bb-8b0e-e05ee79d30c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:53.663531 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2443d1f8-f62f-49bb-8b0e-e05ee79d30c9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:53.670845 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2443d1f8-f62f-49bb-8b0e-e05ee79d30c9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:53.672375 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2443d1f8-f62f-49bb-8b0e-e05ee79d30c9 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:53.685160 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-2443d1f8-f62f-49bb-8b0e-e05ee79d30c9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 11:59:53.685160 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:59:53.685160 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:59:53.685160 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:59:53.685160 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:59:53.685160 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:59:53.685160 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:59:53.685160 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:59:53.685160 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:59:53.685160 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:59:53.685160 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:59:53.685160 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:59:53.685160 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 11:59:53.685160 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 11:59:53.685160 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 11:59:53.685160 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 11:59:53.686055 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:59:53.686055 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:59:53.686055 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 11:59:53.686055 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:59:53.686055 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 11:59:53.686055 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:59:53.686055 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:59:53.686055 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:59:53.686055 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:59:53.686055 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:59:53.686055 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:59:53.686055 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 11:59:53.686055 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:59:53.686055 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:59:53.686055 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:59:53.686055 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:59:53.686055 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:59:53.686055 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:59:53.687399 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:59:53.687399 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:59:53.687399 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 11:59:53.687399 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 11:59:53.687399 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 11:59:53.687399 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 11:59:53.687399 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 11:59:53.687399 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 11:59:53.687399 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 11:59:53.687399 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 11:59:53.687399 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 246/499] 10.222.0.49 () {56 vars in 1093 bytes} [Fri Sep 9 11:59:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:59:53.687399 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dae750a-dd35-4497-a68a-5e53a4f9c297 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:53.688165 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dae750a-dd35-4497-a68a-5e53a4f9c297 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:53.688165 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dae750a-dd35-4497-a68a-5e53a4f9c297 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:53.706184 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-9dae750a-dd35-4497-a68a-5e53a4f9c297 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:59:53.707437 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-9dae750a-dd35-4497-a68a-5e53a4f9c297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:53.718179 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785c13c1-79e5-4b13-84b0-234f70fbac32 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:53.751221 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe92b2f-8636-45ad-a673-73363a5f2a3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:53.751826 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe92b2f-8636-45ad-a673-73363a5f2a3e None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:53.752105 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe92b2f-8636-45ad-a673-73363a5f2a3e None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:53.752378 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe92b2f-8636-45ad-a673-73363a5f2a3e None admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:53.753011 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe92b2f-8636-45ad-a673-73363a5f2a3e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:59:53.753313 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe92b2f-8636-45ad-a673-73363a5f2a3e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:53.755466 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe92b2f-8636-45ad-a673-73363a5f2a3e None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:53.762955 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:53.768120 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 254/500] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 9 11:59:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:53.769022 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe92b2f-8636-45ad-a673-73363a5f2a3e None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:53.777739 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 247/501] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 11:59:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:59:53.779106 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dae750a-dd35-4497-a68a-5e53a4f9c297 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:53.822594 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:53.871619 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe92b2f-8636-45ad-a673-73363a5f2a3e None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:53.902752 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dae750a-dd35-4497-a68a-5e53a4f9c297 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:53.907876 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531654b6-617b-4d43-9ee3-67b47c318115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:53.908504 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531654b6-617b-4d43-9ee3-67b47c318115 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:53.908780 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531654b6-617b-4d43-9ee3-67b47c318115 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:53.909049 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531654b6-617b-4d43-9ee3-67b47c318115 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:53.919510 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531654b6-617b-4d43-9ee3-67b47c318115 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:59:53.924233 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-531654b6-617b-4d43-9ee3-67b47c318115 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 11:59:53.924233 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:59:53.924233 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:59:53.924233 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:59:53.924233 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:59:53.924233 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:59:53.924233 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:59:53.924233 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:59:53.924233 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:59:53.924233 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:59:53.924233 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:59:53.924233 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:59:53.924233 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:59:53.924233 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:59:53.924233 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:59:53.924233 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:59:53.925126 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:59:53.925126 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:59:53.925126 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:59:53.925126 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:59:53.925126 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:59:53.925126 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:59:53.925126 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:59:53.925126 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:59:53.925126 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:59:53.925126 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:59:53.925126 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:59:53.925126 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 11:59:53.925126 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:59:53.925126 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:59:53.925126 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:59:53.925126 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:59:53.925921 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:59:53.925921 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:59:53.925921 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:59:53.925921 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:59:53.925921 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:59:53.925921 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:59:53.925921 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:59:53.925921 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:59:53.925921 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:59:53.925921 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:59:53.925921 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 11:59:53.925921 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 11:59:53.925921 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 255/502] 10.222.0.49 () {56 vars in 1102 bytes} [Fri Sep 9 11:59:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:59:53.927837 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531654b6-617b-4d43-9ee3-67b47c318115 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:53.940743 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fec0b0-aa2d-44cf-9d21-fbb31740e7e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:53.941307 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fec0b0-aa2d-44cf-9d21-fbb31740e7e6 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:53.941579 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fec0b0-aa2d-44cf-9d21-fbb31740e7e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:53.942468 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fec0b0-aa2d-44cf-9d21-fbb31740e7e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:53.953065 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fec0b0-aa2d-44cf-9d21-fbb31740e7e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:53.954505 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fec0b0-aa2d-44cf-9d21-fbb31740e7e6 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:53.969682 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-56fec0b0-aa2d-44cf-9d21-fbb31740e7e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:59:53.969682 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:59:53.969682 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:59:53.969682 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:59:53.969682 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:59:53.969682 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:59:53.969682 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:59:53.969682 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:59:53.969682 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:59:53.969682 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:59:53.969682 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:59:53.969682 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:59:53.969682 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:59:53.969682 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:59:53.969682 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:59:53.969682 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:59:53.970599 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:59:53.970599 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:59:53.970599 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:59:53.970599 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:59:53.970599 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:59:53.970599 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:59:53.970599 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:59:53.970599 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 11:59:53.970599 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:59:53.970599 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:59:53.970599 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:59:53.970599 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:59:53.970599 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:59:53.970599 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:59:53.970599 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:59:53.970599 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:59:53.970599 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:59:53.971470 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:59:53.971470 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:59:53.971470 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 11:59:53.971470 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 248/503] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 11:59:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:59:53.971679 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fec0b0-aa2d-44cf-9d21-fbb31740e7e6 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:53.996705 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:54.041312 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531654b6-617b-4d43-9ee3-67b47c318115 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:54.049457 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:54.077685 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c327e80c-3e3d-47fe-a0f9-b8d7daa374ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:54.078287 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c327e80c-3e3d-47fe-a0f9-b8d7daa374ac None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:54.078570 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c327e80c-3e3d-47fe-a0f9-b8d7daa374ac None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:54.078847 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c327e80c-3e3d-47fe-a0f9-b8d7daa374ac None admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:54.079605 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c327e80c-3e3d-47fe-a0f9-b8d7daa374ac None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:59:54.079905 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c327e80c-3e3d-47fe-a0f9-b8d7daa374ac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:54.081223 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fec0b0-aa2d-44cf-9d21-fbb31740e7e6 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:54.081320 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c327e80c-3e3d-47fe-a0f9-b8d7daa374ac None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:54.114452 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7af9bfc-b387-4122-8c8c-a77cafbb7fca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:54.115107 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7af9bfc-b387-4122-8c8c-a77cafbb7fca None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:54.115384 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7af9bfc-b387-4122-8c8c-a77cafbb7fca None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:54.115646 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7af9bfc-b387-4122-8c8c-a77cafbb7fca None admin] PATH_INFO: `/v3/users/glance` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:54.118561 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 256/504] 10.222.0.49 () {56 vars in 1133 bytes} [Fri Sep 9 11:59:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:54.119413 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c327e80c-3e3d-47fe-a0f9-b8d7daa374ac None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:54.122954 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7af9bfc-b387-4122-8c8c-a77cafbb7fca None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:54.124386 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7af9bfc-b387-4122-8c8c-a77cafbb7fca None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:54.136012 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-c7af9bfc-b387-4122-8c8c-a77cafbb7fca None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 09 11:59:54.136012 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:59:54.136012 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:59:54.136012 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:59:54.136012 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:59:54.136012 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:59:54.136012 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:59:54.136012 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:59:54.136012 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:59:54.136012 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:59:54.136012 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:59:54.136012 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:59:54.136012 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 11:59:54.136012 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 11:59:54.136012 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 11:59:54.136012 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 11:59:54.136924 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:59:54.136924 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:59:54.136924 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 11:59:54.136924 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:59:54.136924 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 11:59:54.136924 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:59:54.136924 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:59:54.136924 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:59:54.136924 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:59:54.136924 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:59:54.136924 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:59:54.136924 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 11:59:54.136924 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:59:54.136924 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:59:54.136924 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:59:54.136924 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:59:54.136924 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:59:54.136924 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:59:54.137805 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:59:54.137805 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:59:54.137805 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 11:59:54.137805 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 11:59:54.137805 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 11:59:54.137805 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 11:59:54.137805 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 11:59:54.137805 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 11:59:54.137805 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 09 11:59:54.137805 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 11:59:54.137805 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 249/505] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 9 11:59:53 2022] GET /identity/v3/users/glance => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:59:54.137805 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7af9bfc-b387-4122-8c8c-a77cafbb7fca None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:54.223751 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c327e80c-3e3d-47fe-a0f9-b8d7daa374ac None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:54.239523 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7af9bfc-b387-4122-8c8c-a77cafbb7fca None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:54.240585 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:54.260036 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22930e87-956d-494a-adcd-b6685c04fcc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:54.260641 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22930e87-956d-494a-adcd-b6685c04fcc8 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:54.260917 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22930e87-956d-494a-adcd-b6685c04fcc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:54.261185 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22930e87-956d-494a-adcd-b6685c04fcc8 None admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:54.261840 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22930e87-956d-494a-adcd-b6685c04fcc8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:59:54.262139 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22930e87-956d-494a-adcd-b6685c04fcc8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:54.263818 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22930e87-956d-494a-adcd-b6685c04fcc8 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:54.268992 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:54.276848 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 257/506] 10.222.0.49 () {56 vars in 1103 bytes} [Fri Sep 9 11:59:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:54.276848 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22930e87-956d-494a-adcd-b6685c04fcc8 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:54.279092 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07100d7e-beae-48a1-b8a6-456390a810c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:54.279699 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07100d7e-beae-48a1-b8a6-456390a810c5 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:54.279974 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07100d7e-beae-48a1-b8a6-456390a810c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:54.280248 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07100d7e-beae-48a1-b8a6-456390a810c5 None admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:54.280885 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07100d7e-beae-48a1-b8a6-456390a810c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:59:54.281184 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07100d7e-beae-48a1-b8a6-456390a810c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:54.282571 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07100d7e-beae-48a1-b8a6-456390a810c5 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:54.295520 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 250/507] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 9 11:59:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:54.296305 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07100d7e-beae-48a1-b8a6-456390a810c5 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:54.377419 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22930e87-956d-494a-adcd-b6685c04fcc8 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:54.396170 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07100d7e-beae-48a1-b8a6-456390a810c5 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:54.410920 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9c4979-1d4b-4c06-81f0-a01fa706713d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:54.411673 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9c4979-1d4b-4c06-81f0-a01fa706713d None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:54.411956 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9c4979-1d4b-4c06-81f0-a01fa706713d None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:54.412230 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9c4979-1d4b-4c06-81f0-a01fa706713d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:54.422892 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9c4979-1d4b-4c06-81f0-a01fa706713d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:54.424356 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9c4979-1d4b-4c06-81f0-a01fa706713d None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:54.429823 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6af128-afe9-4bea-849b-08655ea3b6ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:54.430442 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6af128-afe9-4bea-849b-08655ea3b6ad None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:54.430716 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6af128-afe9-4bea-849b-08655ea3b6ad None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:54.431020 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6af128-afe9-4bea-849b-08655ea3b6ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:54.438931 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-0a9c4979-1d4b-4c06-81f0-a01fa706713d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:59:54.438931 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:59:54.438931 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:59:54.438931 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:59:54.438931 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:59:54.438931 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:59:54.438931 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:59:54.438931 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:59:54.438931 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:59:54.438931 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:59:54.438931 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:59:54.438931 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:59:54.438931 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:59:54.438931 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:59:54.438931 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:59:54.438931 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:59:54.440247 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:59:54.440247 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:59:54.440247 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:59:54.440247 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:59:54.440247 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:59:54.440247 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:59:54.440247 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:59:54.440247 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 11:59:54.440247 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:59:54.440247 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:59:54.440247 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:59:54.440247 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:59:54.440247 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:59:54.440247 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:59:54.440247 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:59:54.440247 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:59:54.440247 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:59:54.441319 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:59:54.441319 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:59:54.441319 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 11:59:54.441319 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 258/508] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 11:59:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:59:54.441319 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9c4979-1d4b-4c06-81f0-a01fa706713d None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:54.470776 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:54.490416 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:54.500178 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6af128-afe9-4bea-849b-08655ea3b6ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:54.501654 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6af128-afe9-4bea-849b-08655ea3b6ad None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:54.539806 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9c4979-1d4b-4c06-81f0-a01fa706713d None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:54.574972 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab203f5-aba5-47ae-9b42-984d0bf2b032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:54.575583 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab203f5-aba5-47ae-9b42-984d0bf2b032 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:54.575861 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab203f5-aba5-47ae-9b42-984d0bf2b032 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:54.576134 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab203f5-aba5-47ae-9b42-984d0bf2b032 None admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:54.576861 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab203f5-aba5-47ae-9b42-984d0bf2b032 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:59:54.577169 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab203f5-aba5-47ae-9b42-984d0bf2b032 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:54.578963 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab203f5-aba5-47ae-9b42-984d0bf2b032 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:54.596069 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 259/509] 10.222.0.49 () {56 vars in 1137 bytes} [Fri Sep 9 11:59:54 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:54.596885 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab203f5-aba5-47ae-9b42-984d0bf2b032 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:54.600196 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 251/510] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 9 11:59:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 11:59:54.600876 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6af128-afe9-4bea-849b-08655ea3b6ad None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:54.690952 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:54.694930 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:54.704386 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6af128-afe9-4bea-849b-08655ea3b6ad None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:54.704931 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab203f5-aba5-47ae-9b42-984d0bf2b032 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:54.737487 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bc5221-a573-41b5-a9a7-9e9a2a5c0df1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:54.737971 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d11a36-700b-422f-b1eb-208024e33ece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:54.738112 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bc5221-a573-41b5-a9a7-9e9a2a5c0df1 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:54.738387 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bc5221-a573-41b5-a9a7-9e9a2a5c0df1 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:54.739131 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d11a36-700b-422f-b1eb-208024e33ece None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:54.739268 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bc5221-a573-41b5-a9a7-9e9a2a5c0df1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:54.739429 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d11a36-700b-422f-b1eb-208024e33ece None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:54.739700 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d11a36-700b-422f-b1eb-208024e33ece None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:54.747374 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d11a36-700b-422f-b1eb-208024e33ece None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:54.749122 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d11a36-700b-422f-b1eb-208024e33ece None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:54.749377 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bc5221-a573-41b5-a9a7-9e9a2a5c0df1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:54.750822 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bc5221-a573-41b5-a9a7-9e9a2a5c0df1 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:54.761040 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-04d11a36-700b-422f-b1eb-208024e33ece None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 11:59:54.761040 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:59:54.761040 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:59:54.761040 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:59:54.761040 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:59:54.761040 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:59:54.761040 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:59:54.761040 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:59:54.761040 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:59:54.761040 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:59:54.761040 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:59:54.761040 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:59:54.761040 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 11:59:54.761040 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 11:59:54.761040 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 11:59:54.761040 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 11:59:54.761956 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:59:54.761956 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:59:54.761956 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 11:59:54.761956 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:59:54.761956 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 11:59:54.761956 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:59:54.761956 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:59:54.761956 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:59:54.761956 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:59:54.761956 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:59:54.761956 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:59:54.761956 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 11:59:54.761956 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:59:54.761956 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:59:54.761956 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:59:54.761956 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:59:54.761956 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:59:54.761956 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:59:54.762846 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:59:54.762846 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:59:54.762846 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 11:59:54.762846 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 11:59:54.762846 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 11:59:54.762846 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 11:59:54.762846 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 11:59:54.762846 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 11:59:54.762846 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 11:59:54.762846 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 11:59:54.762846 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 252/511] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 9 11:59:54 2022] GET /identity/v3/users/cinder => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:59:54.762846 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d11a36-700b-422f-b1eb-208024e33ece None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:54.765902 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-98bc5221-a573-41b5-a9a7-9e9a2a5c0df1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:59:54.765902 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:59:54.765902 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:59:54.765902 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:59:54.765902 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:59:54.765902 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:59:54.765902 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:59:54.765902 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:59:54.765902 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:59:54.765902 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:59:54.765902 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:59:54.765902 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:59:54.765902 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:59:54.765902 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:59:54.765902 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:59:54.765902 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:59:54.766875 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:59:54.766875 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:59:54.766875 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:59:54.766875 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:59:54.766875 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:59:54.766875 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:59:54.766875 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:59:54.766875 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 11:59:54.766875 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:59:54.766875 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:59:54.766875 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:59:54.766875 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:59:54.766875 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:59:54.766875 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:59:54.766875 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:59:54.766875 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:59:54.766875 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:59:54.768971 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:59:54.768971 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:59:54.768971 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 11:59:54.768971 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 260/512] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 11:59:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:59:54.768971 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bc5221-a573-41b5-a9a7-9e9a2a5c0df1 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:54.869038 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d11a36-700b-422f-b1eb-208024e33ece None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:54.869953 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bc5221-a573-41b5-a9a7-9e9a2a5c0df1 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:54.901635 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72192223-d7af-474f-a4d9-dd55e97a7d6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:54.902233 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72192223-d7af-474f-a4d9-dd55e97a7d6a None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:54.902540 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72192223-d7af-474f-a4d9-dd55e97a7d6a None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:54.902808 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72192223-d7af-474f-a4d9-dd55e97a7d6a None admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:54.903471 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72192223-d7af-474f-a4d9-dd55e97a7d6a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:59:54.903570 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b9ab19-b539-4cb4-ab4e-38d5d6a499d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:54.903988 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72192223-d7af-474f-a4d9-dd55e97a7d6a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:54.904088 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b9ab19-b539-4cb4-ab4e-38d5d6a499d3 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:54.904333 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b9ab19-b539-4cb4-ab4e-38d5d6a499d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:54.904597 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b9ab19-b539-4cb4-ab4e-38d5d6a499d3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:54.905144 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72192223-d7af-474f-a4d9-dd55e97a7d6a None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:54.915751 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:54.917318 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 253/513] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 9 11:59:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:54.918125 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72192223-d7af-474f-a4d9-dd55e97a7d6a None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:54.934071 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:54.974820 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b9ab19-b539-4cb4-ab4e-38d5d6a499d3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:54.976293 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b9ab19-b539-4cb4-ab4e-38d5d6a499d3 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:55.019226 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72192223-d7af-474f-a4d9-dd55e97a7d6a None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:55.052585 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789ea144-3459-4986-ae1c-9a31d3f52445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:55.053207 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789ea144-3459-4986-ae1c-9a31d3f52445 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:55.053481 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789ea144-3459-4986-ae1c-9a31d3f52445 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:55.053743 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789ea144-3459-4986-ae1c-9a31d3f52445 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:55.063348 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789ea144-3459-4986-ae1c-9a31d3f52445 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:55.065048 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789ea144-3459-4986-ae1c-9a31d3f52445 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:55.073267 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 261/514] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 9 11:59:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 11:59:55.074111 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b9ab19-b539-4cb4-ab4e-38d5d6a499d3 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:55.084870 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-789ea144-3459-4986-ae1c-9a31d3f52445 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:59:55.084870 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:59:55.084870 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:59:55.084870 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:59:55.084870 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:59:55.084870 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:59:55.084870 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:59:55.084870 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:59:55.084870 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:59:55.084870 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:59:55.084870 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:59:55.084870 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:59:55.084870 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:59:55.084870 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:59:55.084870 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:59:55.084870 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:59:55.085888 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:59:55.085888 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:59:55.085888 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:59:55.085888 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:59:55.085888 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:59:55.085888 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:59:55.085888 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:59:55.085888 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 11:59:55.085888 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:59:55.085888 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:59:55.085888 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:59:55.085888 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:59:55.085888 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:59:55.085888 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:59:55.085888 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:59:55.085888 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:59:55.085888 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:59:55.086740 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:59:55.086740 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:59:55.086740 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 11:59:55.086740 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 254/515] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 11:59:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:59:55.086740 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789ea144-3459-4986-ae1c-9a31d3f52445 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:55.118070 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:55.148844 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:55.173973 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b9ab19-b539-4cb4-ab4e-38d5d6a499d3 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:55.187630 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789ea144-3459-4986-ae1c-9a31d3f52445 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:55.208193 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0247730-973e-423b-b946-7b701e9abe1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:55.208788 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0247730-973e-423b-b946-7b701e9abe1b None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:55.209059 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0247730-973e-423b-b946-7b701e9abe1b None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:55.209319 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0247730-973e-423b-b946-7b701e9abe1b None admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:55.210028 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0247730-973e-423b-b946-7b701e9abe1b None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:59:55.210319 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0247730-973e-423b-b946-7b701e9abe1b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:55.211758 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0247730-973e-423b-b946-7b701e9abe1b None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:55.221098 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce8e71d-9957-4f68-b508-cde447f50c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:55.221697 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce8e71d-9957-4f68-b508-cde447f50c88 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:55.221978 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce8e71d-9957-4f68-b508-cde447f50c88 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:55.222251 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce8e71d-9957-4f68-b508-cde447f50c88 None admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:55.222882 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce8e71d-9957-4f68-b508-cde447f50c88 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:59:55.223255 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce8e71d-9957-4f68-b508-cde447f50c88 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:55.224621 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce8e71d-9957-4f68-b508-cde447f50c88 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:55.228264 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 262/516] 10.222.0.49 () {56 vars in 1137 bytes} [Fri Sep 9 11:59:55 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:55.229039 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0247730-973e-423b-b946-7b701e9abe1b None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:55.241257 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 255/517] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 9 11:59:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:55.241257 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce8e71d-9957-4f68-b508-cde447f50c88 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:55.324341 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:55.328569 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0247730-973e-423b-b946-7b701e9abe1b None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:55.337548 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce8e71d-9957-4f68-b508-cde447f50c88 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:55.360335 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:55.361954 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22633958-a2e8-4840-9f5a-64845faf4537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:55.362621 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22633958-a2e8-4840-9f5a-64845faf4537 None admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:55.362893 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22633958-a2e8-4840-9f5a-64845faf4537 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:55.363191 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22633958-a2e8-4840-9f5a-64845faf4537 None admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:55.364460 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22633958-a2e8-4840-9f5a-64845faf4537 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:55.365990 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22633958-a2e8-4840-9f5a-64845faf4537 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:55.371341 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c6a4c7-9c48-4dea-8ecd-7482b917e1ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:55.371955 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c6a4c7-9c48-4dea-8ecd-7482b917e1ab None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:55.372229 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c6a4c7-9c48-4dea-8ecd-7482b917e1ab None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:55.372495 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c6a4c7-9c48-4dea-8ecd-7482b917e1ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:55.383128 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c6a4c7-9c48-4dea-8ecd-7482b917e1ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:55.384557 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c6a4c7-9c48-4dea-8ecd-7482b917e1ab None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:55.386338 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 263/518] 10.222.0.49 () {60 vars in 1132 bytes} [Fri Sep 9 11:59:55 2022] POST /identity/v3/users => generated 247 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 11:59:55.387188 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22633958-a2e8-4840-9f5a-64845faf4537 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:55.400589 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-51c6a4c7-9c48-4dea-8ecd-7482b917e1ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:59:55.400589 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:59:55.400589 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:59:55.400589 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:59:55.400589 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:59:55.400589 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:59:55.400589 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:59:55.400589 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:59:55.400589 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:59:55.400589 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:59:55.400589 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:59:55.400589 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:59:55.400589 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:59:55.400589 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:59:55.400589 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:59:55.400589 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:59:55.401763 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:59:55.401763 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:59:55.401763 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:59:55.401763 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:59:55.401763 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:59:55.401763 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:59:55.401763 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:59:55.401763 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 11:59:55.401763 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:59:55.401763 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:59:55.401763 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:59:55.401763 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:59:55.401763 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:59:55.401763 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:59:55.401763 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:59:55.401763 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:59:55.401763 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:59:55.402823 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:59:55.402823 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:59:55.402823 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 11:59:55.403356 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 256/519] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 11:59:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:59:55.404239 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c6a4c7-9c48-4dea-8ecd-7482b917e1ab None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:55.519142 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22633958-a2e8-4840-9f5a-64845faf4537 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:55.521575 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c6a4c7-9c48-4dea-8ecd-7482b917e1ab None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:55.557475 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa84434-ea71-4a7d-904a-ef51c01bdffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:55.558088 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa84434-ea71-4a7d-904a-ef51c01bdffa None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:55.558377 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa84434-ea71-4a7d-904a-ef51c01bdffa None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:55.558638 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa84434-ea71-4a7d-904a-ef51c01bdffa None admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:55.559320 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa84434-ea71-4a7d-904a-ef51c01bdffa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:59:55.559632 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa84434-ea71-4a7d-904a-ef51c01bdffa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:55.561015 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa84434-ea71-4a7d-904a-ef51c01bdffa None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:55.561556 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de28fe77-75c0-42d8-817f-1df3a43727c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:55.562275 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de28fe77-75c0-42d8-817f-1df3a43727c3 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:55.562643 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de28fe77-75c0-42d8-817f-1df3a43727c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:55.563034 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de28fe77-75c0-42d8-817f-1df3a43727c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:55.576347 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 264/520] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 9 11:59:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:55.577718 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa84434-ea71-4a7d-904a-ef51c01bdffa None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:55.593857 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:55.617545 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:55.649339 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de28fe77-75c0-42d8-817f-1df3a43727c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:55.650835 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de28fe77-75c0-42d8-817f-1df3a43727c3 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:55.696414 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa84434-ea71-4a7d-904a-ef51c01bdffa None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:55.734819 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91da449e-b9fc-437a-9ba7-e9e0cde671e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:55.735482 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91da449e-b9fc-437a-9ba7-e9e0cde671e8 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:55.735761 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91da449e-b9fc-437a-9ba7-e9e0cde671e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:55.736029 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91da449e-b9fc-437a-9ba7-e9e0cde671e8 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:55.744207 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91da449e-b9fc-437a-9ba7-e9e0cde671e8 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:55.745718 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91da449e-b9fc-437a-9ba7-e9e0cde671e8 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:55.759111 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 257/521] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 9 11:59:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 11:59:55.759292 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-91da449e-b9fc-437a-9ba7-e9e0cde671e8 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 09 11:59:55.759292 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:59:55.759292 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:59:55.759292 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:59:55.759292 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:59:55.759292 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:59:55.759292 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:59:55.759292 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:59:55.759292 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:59:55.759292 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:59:55.759292 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:59:55.759292 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:59:55.759292 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 11:59:55.759292 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 11:59:55.759292 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 11:59:55.759292 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 11:59:55.760092 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:59:55.760092 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:59:55.760092 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 11:59:55.760092 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:59:55.760092 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 11:59:55.760092 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:59:55.760092 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:59:55.760092 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:59:55.760092 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:59:55.760092 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:59:55.760092 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:59:55.760092 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 11:59:55.760092 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:59:55.760092 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:59:55.760092 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:59:55.760092 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:59:55.760092 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:59:55.760092 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:59:55.761001 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:59:55.761001 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:59:55.761001 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 11:59:55.761001 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 11:59:55.761001 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 11:59:55.761001 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 11:59:55.761001 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 11:59:55.761001 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 11:59:55.761001 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 09 11:59:55.761001 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 11:59:55.762912 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de28fe77-75c0-42d8-817f-1df3a43727c3 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:55.764811 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 265/522] 10.222.0.49 () {56 vars in 1102 bytes} [Fri Sep 9 11:59:55 2022] GET /identity/v3/users/neutron => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:59:55.765666 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91da449e-b9fc-437a-9ba7-e9e0cde671e8 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:55.845896 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:55.846815 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:55.872724 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de28fe77-75c0-42d8-817f-1df3a43727c3 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:55.873360 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91da449e-b9fc-437a-9ba7-e9e0cde671e8 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:55.909943 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5581e229-1cbf-47e3-9b46-a51aef1bac80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:55.910590 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5581e229-1cbf-47e3-9b46-a51aef1bac80 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:55.911001 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5581e229-1cbf-47e3-9b46-a51aef1bac80 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:55.911408 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5581e229-1cbf-47e3-9b46-a51aef1bac80 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:55.914736 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3445262c-05a6-40eb-b6b3-c49e4f0347a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:55.916854 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3445262c-05a6-40eb-b6b3-c49e4f0347a5 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:55.916854 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3445262c-05a6-40eb-b6b3-c49e4f0347a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:55.916854 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3445262c-05a6-40eb-b6b3-c49e4f0347a5 None admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:55.916854 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3445262c-05a6-40eb-b6b3-c49e4f0347a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:59:55.916854 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3445262c-05a6-40eb-b6b3-c49e4f0347a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:55.918196 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3445262c-05a6-40eb-b6b3-c49e4f0347a5 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:55.918397 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5581e229-1cbf-47e3-9b46-a51aef1bac80 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:55.919876 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5581e229-1cbf-47e3-9b46-a51aef1bac80 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:55.934948 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 258/523] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 9 11:59:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:55.934948 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3445262c-05a6-40eb-b6b3-c49e4f0347a5 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:55.935178 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-5581e229-1cbf-47e3-9b46-a51aef1bac80 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:59:55.935178 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:59:55.935178 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:59:55.935178 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:59:55.935178 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:59:55.935178 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:59:55.935178 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:59:55.935178 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:59:55.935178 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:59:55.935178 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:59:55.935178 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:59:55.935178 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:59:55.935178 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 11:59:55.935178 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 11:59:55.935178 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 11:59:55.935972 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 11:59:55.935972 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:59:55.935972 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:59:55.935972 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:59:55.935972 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:59:55.935972 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:59:55.935972 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:59:55.935972 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:59:55.935972 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 11:59:55.935972 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:59:55.935972 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:59:55.935972 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:59:55.935972 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:59:55.935972 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:59:55.935972 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:59:55.935972 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:59:55.935972 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:59:55.936859 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 11:59:55.936859 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 11:59:55.936859 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 11:59:55.936859 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 11:59:55.936859 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 11:59:55.936859 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 11:59:55.936859 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:59:55.936859 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 11:59:55.936859 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 266/524] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 9 11:59:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:59:55.936859 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5581e229-1cbf-47e3-9b46-a51aef1bac80 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:56.045896 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3445262c-05a6-40eb-b6b3-c49e4f0347a5 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:56.047192 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5581e229-1cbf-47e3-9b46-a51aef1bac80 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:56.082859 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b82e84a-b936-4e72-b94d-a9f93738b1dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:56.083532 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b82e84a-b936-4e72-b94d-a9f93738b1dd None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:56.083805 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b82e84a-b936-4e72-b94d-a9f93738b1dd None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:56.084020 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92dcbaf-51eb-44be-ab88-deb875c38d0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:56.084132 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b82e84a-b936-4e72-b94d-a9f93738b1dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:56.084641 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92dcbaf-51eb-44be-ab88-deb875c38d0e None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:56.084891 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92dcbaf-51eb-44be-ab88-deb875c38d0e None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:56.085154 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92dcbaf-51eb-44be-ab88-deb875c38d0e None admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:56.086052 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92dcbaf-51eb-44be-ab88-deb875c38d0e None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:59:56.086364 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92dcbaf-51eb-44be-ab88-deb875c38d0e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:56.086852 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:56.088214 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92dcbaf-51eb-44be-ab88-deb875c38d0e None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:56.090107 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:56.097477 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b82e84a-b936-4e72-b94d-a9f93738b1dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:56.099001 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b82e84a-b936-4e72-b94d-a9f93738b1dd None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:56.108743 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 267/525] 10.222.0.49 () {56 vars in 1139 bytes} [Fri Sep 9 11:59:55 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:56.115302 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92dcbaf-51eb-44be-ab88-deb875c38d0e None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:56.115442 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-7b82e84a-b936-4e72-b94d-a9f93738b1dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:59:56.115442 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:59:56.115442 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:59:56.115442 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:59:56.115442 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:59:56.115442 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:59:56.115442 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:59:56.115442 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:59:56.115442 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:59:56.115442 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:59:56.115442 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:59:56.115442 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:59:56.115442 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:59:56.115442 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:59:56.115442 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:59:56.115442 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:59:56.120897 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:59:56.120897 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:59:56.120897 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:59:56.120897 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:59:56.120897 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:59:56.120897 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:59:56.120897 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:59:56.120897 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 11:59:56.120897 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:59:56.120897 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:59:56.120897 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:59:56.120897 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:59:56.120897 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:59:56.120897 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:59:56.120897 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:59:56.120897 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:59:56.120897 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:59:56.121944 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:59:56.121944 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:59:56.121944 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 11:59:56.121944 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 259/526] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 11:59:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:59:56.121944 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b82e84a-b936-4e72-b94d-a9f93738b1dd None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:56.227911 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92dcbaf-51eb-44be-ab88-deb875c38d0e None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:56.229266 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b82e84a-b936-4e72-b94d-a9f93738b1dd None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:56.263276 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fc76f4-0961-4160-809a-55d05df5e58a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:56.264134 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fc76f4-0961-4160-809a-55d05df5e58a None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:56.264573 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fc76f4-0961-4160-809a-55d05df5e58a None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:56.264984 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fc76f4-0961-4160-809a-55d05df5e58a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:56.272583 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fc76f4-0961-4160-809a-55d05df5e58a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:56.273755 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a866ad16-9982-4bc1-8930-92f106f58b86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:56.274363 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a866ad16-9982-4bc1-8930-92f106f58b86 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:56.274544 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fc76f4-0961-4160-809a-55d05df5e58a None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:56.274649 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a866ad16-9982-4bc1-8930-92f106f58b86 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:56.274938 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a866ad16-9982-4bc1-8930-92f106f58b86 None admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:56.275627 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a866ad16-9982-4bc1-8930-92f106f58b86 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:59:56.276000 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a866ad16-9982-4bc1-8930-92f106f58b86 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:56.277448 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a866ad16-9982-4bc1-8930-92f106f58b86 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:56.287648 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-84fc76f4-0961-4160-809a-55d05df5e58a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:59:56.287648 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:59:56.287648 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:59:56.287648 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:59:56.287648 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:59:56.287648 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:59:56.287648 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:59:56.287648 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:59:56.287648 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:59:56.287648 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:59:56.287648 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:59:56.287648 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:59:56.287648 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 11:59:56.287648 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 11:59:56.287648 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 11:59:56.288654 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 11:59:56.288654 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:59:56.288654 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:59:56.288654 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:59:56.288654 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:59:56.288654 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:59:56.288654 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:59:56.288654 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:59:56.288654 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 11:59:56.288654 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:59:56.288654 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:59:56.288654 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:59:56.288654 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:59:56.288654 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:59:56.288654 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:59:56.288654 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:59:56.288654 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:59:56.289719 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 11:59:56.289719 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 11:59:56.289719 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 11:59:56.289719 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 11:59:56.289719 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 11:59:56.289719 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 11:59:56.289719 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:59:56.289719 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 11:59:56.291422 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 268/527] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 9 11:59:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:59:56.296911 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fc76f4-0961-4160-809a-55d05df5e58a None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:56.297292 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 260/528] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Sep 9 11:59:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:56.298099 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a866ad16-9982-4bc1-8930-92f106f58b86 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:56.351329 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:56.360947 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:56.403163 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a866ad16-9982-4bc1-8930-92f106f58b86 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:56.415128 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fc76f4-0961-4160-809a-55d05df5e58a None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:56.438883 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d62510-108c-4f0d-b99c-9851d5921779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:56.439930 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d62510-108c-4f0d-b99c-9851d5921779 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:56.440210 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d62510-108c-4f0d-b99c-9851d5921779 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:56.440796 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d62510-108c-4f0d-b99c-9851d5921779 None admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:56.442209 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d62510-108c-4f0d-b99c-9851d5921779 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:59:56.442516 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d62510-108c-4f0d-b99c-9851d5921779 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:56.444440 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d62510-108c-4f0d-b99c-9851d5921779 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:56.451743 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55982d68-92c8-4af5-a040-184bda725fd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:56.452339 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55982d68-92c8-4af5-a040-184bda725fd8 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:56.452608 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55982d68-92c8-4af5-a040-184bda725fd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:56.452869 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55982d68-92c8-4af5-a040-184bda725fd8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:56.463134 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 261/529] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 9 11:59:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:56.463134 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d62510-108c-4f0d-b99c-9851d5921779 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:56.524292 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55982d68-92c8-4af5-a040-184bda725fd8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:56.525899 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55982d68-92c8-4af5-a040-184bda725fd8 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:56.581335 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d62510-108c-4f0d-b99c-9851d5921779 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:56.584381 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:56.595006 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:56.628184 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca1ea8a-72e5-4999-bb02-c8390f7f87f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:56.629064 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca1ea8a-72e5-4999-bb02-c8390f7f87f8 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:56.629695 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca1ea8a-72e5-4999-bb02-c8390f7f87f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:56.630110 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca1ea8a-72e5-4999-bb02-c8390f7f87f8 None admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:56.631030 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca1ea8a-72e5-4999-bb02-c8390f7f87f8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:59:56.631331 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca1ea8a-72e5-4999-bb02-c8390f7f87f8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:56.632764 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca1ea8a-72e5-4999-bb02-c8390f7f87f8 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:56.637371 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 269/530] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 9 11:59:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 11:59:56.643075 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55982d68-92c8-4af5-a040-184bda725fd8 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:56.646323 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 262/531] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Sep 9 11:59:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:56.647096 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca1ea8a-72e5-4999-bb02-c8390f7f87f8 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:56.749059 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55982d68-92c8-4af5-a040-184bda725fd8 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:56.758026 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca1ea8a-72e5-4999-bb02-c8390f7f87f8 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:56.786398 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29a6559-232a-444e-931f-2818f6643ad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:56.786845 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29a6559-232a-444e-931f-2818f6643ad1 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:56.787164 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29a6559-232a-444e-931f-2818f6643ad1 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:56.787454 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29a6559-232a-444e-931f-2818f6643ad1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:56.788149 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29a6559-232a-444e-931f-2818f6643ad1 None admin] RBAC: Adding query filter params (role.id=8abeff2cb87d4b7fa72b27b028005877, scope.project.id=fe863fcd75f1406ebbffc5016078f73e, user.id=70bac169d5cd43e1926f310344ac11fa) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:59:56.788454 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29a6559-232a-444e-931f-2818f6643ad1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:56.790649 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29a6559-232a-444e-931f-2818f6643ad1 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:56.794404 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64d06ed-2293-4374-9a4f-5147a246b23d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:56.794956 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64d06ed-2293-4374-9a4f-5147a246b23d None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:56.795232 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64d06ed-2293-4374-9a4f-5147a246b23d None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:56.795497 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64d06ed-2293-4374-9a4f-5147a246b23d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:56.803084 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 270/532] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Sep 9 11:59:56 2022] GET /identity/v3/role_assignments?user.id=70bac169d5cd43e1926f310344ac11fa&scope.project.id=fe863fcd75f1406ebbffc5016078f73e&role.id=8abeff2cb87d4b7fa72b27b028005877 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:56.803084 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29a6559-232a-444e-931f-2818f6643ad1 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:56.809402 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64d06ed-2293-4374-9a4f-5147a246b23d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:56.811095 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64d06ed-2293-4374-9a4f-5147a246b23d None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:56.823952 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:56.825421 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-b64d06ed-2293-4374-9a4f-5147a246b23d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:59:56.825421 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:59:56.825421 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:59:56.825421 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:59:56.825421 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:59:56.825421 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:59:56.825421 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:59:56.825421 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:59:56.825421 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:59:56.825421 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:59:56.825421 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:59:56.825421 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:59:56.825421 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 11:59:56.825421 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 11:59:56.825421 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 11:59:56.826274 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 11:59:56.826274 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:59:56.826274 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:59:56.826274 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:59:56.826274 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:59:56.826274 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:59:56.826274 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:59:56.826274 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:59:56.826274 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 11:59:56.826274 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:59:56.826274 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:59:56.826274 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:59:56.826274 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:59:56.826274 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:59:56.826274 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:59:56.826274 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:59:56.826274 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:59:56.827145 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 11:59:56.827145 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 11:59:56.827145 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 11:59:56.827145 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 11:59:56.827145 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 11:59:56.827145 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 11:59:56.827145 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:59:56.827145 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 11:59:56.827145 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 263/533] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 9 11:59:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:59:56.828136 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64d06ed-2293-4374-9a4f-5147a246b23d None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:56.869160 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:56.945685 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29a6559-232a-444e-931f-2818f6643ad1 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:56.956160 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64d06ed-2293-4374-9a4f-5147a246b23d None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:56.997004 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9967ae6-3c27-42b7-b8bf-41ac3d357de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:56.997644 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9967ae6-3c27-42b7-b8bf-41ac3d357de4 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:56.997915 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9967ae6-3c27-42b7-b8bf-41ac3d357de4 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:56.998173 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9967ae6-3c27-42b7-b8bf-41ac3d357de4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:56.999288 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0364c6-e61d-4952-a7e9-a888e5fe3a28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:57.000393 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0364c6-e61d-4952-a7e9-a888e5fe3a28 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:57.000393 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0364c6-e61d-4952-a7e9-a888e5fe3a28 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:57.000393 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0364c6-e61d-4952-a7e9-a888e5fe3a28 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:57.001452 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0364c6-e61d-4952-a7e9-a888e5fe3a28 None admin] RBAC: Adding query filter params (role.id=8abeff2cb87d4b7fa72b27b028005877, scope.project.id=fe863fcd75f1406ebbffc5016078f73e, user.id=37c4e14aa8e04d3e85a406b9363bca2a) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:59:57.001452 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0364c6-e61d-4952-a7e9-a888e5fe3a28 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:57.002790 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0364c6-e61d-4952-a7e9-a888e5fe3a28 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:57.010011 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 264/534] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Sep 9 11:59:56 2022] GET /identity/v3/role_assignments?user.id=37c4e14aa8e04d3e85a406b9363bca2a&scope.project.id=fe863fcd75f1406ebbffc5016078f73e&role.id=8abeff2cb87d4b7fa72b27b028005877 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:57.010841 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0364c6-e61d-4952-a7e9-a888e5fe3a28 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:57.030474 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9967ae6-3c27-42b7-b8bf-41ac3d357de4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:57.032105 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9967ae6-3c27-42b7-b8bf-41ac3d357de4 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:57.055293 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-b9967ae6-3c27-42b7-b8bf-41ac3d357de4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:59:57.055293 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:59:57.055293 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:59:57.055293 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:59:57.055293 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:59:57.055293 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:59:57.055293 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:59:57.055293 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:59:57.055293 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:59:57.055293 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:59:57.055293 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:59:57.055293 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:59:57.055293 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:59:57.055293 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:59:57.055293 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:59:57.055293 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:59:57.056262 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:59:57.056262 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:59:57.056262 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:59:57.056262 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:59:57.056262 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:59:57.056262 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:59:57.056262 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:59:57.056262 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 11:59:57.056262 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:59:57.056262 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:59:57.056262 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:59:57.056262 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:59:57.056262 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:59:57.056262 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:59:57.056262 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:59:57.056262 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:59:57.056262 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:59:57.057117 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:59:57.057117 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:59:57.057117 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 11:59:57.059047 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 271/535] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 11:59:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:59:57.065148 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9967ae6-3c27-42b7-b8bf-41ac3d357de4 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:57.188447 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:57.198828 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0364c6-e61d-4952-a7e9-a888e5fe3a28 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:57.230193 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:57.250230 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c00d6f-1af4-4cdd-96e1-15eee643679d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:57.250846 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c00d6f-1af4-4cdd-96e1-15eee643679d None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:57.251138 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c00d6f-1af4-4cdd-96e1-15eee643679d None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:57.251400 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c00d6f-1af4-4cdd-96e1-15eee643679d None admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:57.252105 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c00d6f-1af4-4cdd-96e1-15eee643679d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:59:57.252389 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c00d6f-1af4-4cdd-96e1-15eee643679d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:57.253861 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c00d6f-1af4-4cdd-96e1-15eee643679d None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:57.279596 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 265/536] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Sep 9 11:59:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:57.288120 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c00d6f-1af4-4cdd-96e1-15eee643679d None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:57.297827 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9967ae6-3c27-42b7-b8bf-41ac3d357de4 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:57.362369 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a9c713-5e56-4d8f-9c34-a060bcfd359d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:57.363155 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a9c713-5e56-4d8f-9c34-a060bcfd359d None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:57.363438 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a9c713-5e56-4d8f-9c34-a060bcfd359d None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:57.363762 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a9c713-5e56-4d8f-9c34-a060bcfd359d None admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:57.364526 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a9c713-5e56-4d8f-9c34-a060bcfd359d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:59:57.364829 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a9c713-5e56-4d8f-9c34-a060bcfd359d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:57.366194 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a9c713-5e56-4d8f-9c34-a060bcfd359d None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:57.396875 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 272/537] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 9 11:59:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:57.403714 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a9c713-5e56-4d8f-9c34-a060bcfd359d None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:57.417203 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c00d6f-1af4-4cdd-96e1-15eee643679d None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:57.484465 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ab5425-de62-4f1b-9ecb-955a2bb41bd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:57.485029 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ab5425-de62-4f1b-9ecb-955a2bb41bd5 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:57.485304 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ab5425-de62-4f1b-9ecb-955a2bb41bd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:57.485568 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ab5425-de62-4f1b-9ecb-955a2bb41bd5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:57.486512 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ab5425-de62-4f1b-9ecb-955a2bb41bd5 None admin] RBAC: Adding query filter params (role.id=8abeff2cb87d4b7fa72b27b028005877, scope.project.id=fe863fcd75f1406ebbffc5016078f73e, user.id=535c258100df40b58a5ee44b2c21d621) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:59:57.486817 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ab5425-de62-4f1b-9ecb-955a2bb41bd5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:57.488277 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ab5425-de62-4f1b-9ecb-955a2bb41bd5 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:57.502863 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 266/538] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Sep 9 11:59:57 2022] GET /identity/v3/role_assignments?user.id=535c258100df40b58a5ee44b2c21d621&scope.project.id=fe863fcd75f1406ebbffc5016078f73e&role.id=8abeff2cb87d4b7fa72b27b028005877 => generated 259 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:59:57.563623 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a9c713-5e56-4d8f-9c34-a060bcfd359d None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:57.595749 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:57.606310 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df963a9-3e50-4d59-8a86-5d73c0fcb96c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:57.607333 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df963a9-3e50-4d59-8a86-5d73c0fcb96c None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:57.607847 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df963a9-3e50-4d59-8a86-5d73c0fcb96c None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:57.608328 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df963a9-3e50-4d59-8a86-5d73c0fcb96c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:57.616913 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df963a9-3e50-4d59-8a86-5d73c0fcb96c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:57.618732 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df963a9-3e50-4d59-8a86-5d73c0fcb96c None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:57.632211 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-7df963a9-3e50-4d59-8a86-5d73c0fcb96c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:59:57.632211 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:59:57.632211 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:59:57.632211 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:59:57.632211 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:59:57.632211 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:59:57.632211 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:59:57.632211 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:59:57.632211 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:59:57.632211 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:59:57.632211 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:59:57.632211 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:59:57.632211 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 11:59:57.632211 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 11:59:57.632211 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 11:59:57.633109 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 11:59:57.633109 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:59:57.633109 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:59:57.633109 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:59:57.633109 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:59:57.633109 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:59:57.633109 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:59:57.633109 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:59:57.633109 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 11:59:57.633109 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:59:57.633109 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:59:57.633109 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:59:57.633109 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:59:57.633109 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:59:57.633109 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:59:57.633109 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:59:57.633109 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:59:57.633946 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 11:59:57.633946 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 11:59:57.633946 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 11:59:57.633946 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 11:59:57.633946 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 11:59:57.633946 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 11:59:57.633946 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 11:59:57.633946 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 11:59:57.636431 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 273/539] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 9 11:59:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 11:59:57.646548 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ab5425-de62-4f1b-9ecb-955a2bb41bd5 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:57.672855 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a6b1f4-eae1-4041-9f53-cfac5d12d139 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:57.673479 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a6b1f4-eae1-4041-9f53-cfac5d12d139 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:57.673953 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a6b1f4-eae1-4041-9f53-cfac5d12d139 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:57.675727 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 274/540] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 11:59:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:59:57.682578 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932e2651-2d10-4ba1-b7b6-ae7b9ed55a4d None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:57.682750 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:57.683304 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932e2651-2d10-4ba1-b7b6-ae7b9ed55a4d None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:57.683792 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932e2651-2d10-4ba1-b7b6-ae7b9ed55a4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:57.713071 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-932e2651-2d10-4ba1-b7b6-ae7b9ed55a4d None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:59:57.714625 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-932e2651-2d10-4ba1-b7b6-ae7b9ed55a4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:57.793909 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 275/541] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 11:59:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 11:59:57.839377 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ab5425-de62-4f1b-9ecb-955a2bb41bd5 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:57.874872 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc45bb56-e5f6-4b7c-9718-858e03c41655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:57.875816 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc45bb56-e5f6-4b7c-9718-858e03c41655 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:57.876305 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc45bb56-e5f6-4b7c-9718-858e03c41655 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:57.876779 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc45bb56-e5f6-4b7c-9718-858e03c41655 None admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:57.877645 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc45bb56-e5f6-4b7c-9718-858e03c41655 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:59:57.878158 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc45bb56-e5f6-4b7c-9718-858e03c41655 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:57.880653 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc45bb56-e5f6-4b7c-9718-858e03c41655 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:57.894166 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 267/542] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Sep 9 11:59:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:57.900597 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932e2651-2d10-4ba1-b7b6-ae7b9ed55a4d None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:58.042132 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932e2651-2d10-4ba1-b7b6-ae7b9ed55a4d None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:58.060778 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:58.080611 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897eb3c5-8eca-4bac-94f2-3b77c3cfac55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:58.081644 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897eb3c5-8eca-4bac-94f2-3b77c3cfac55 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:58.082116 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897eb3c5-8eca-4bac-94f2-3b77c3cfac55 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:58.083501 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897eb3c5-8eca-4bac-94f2-3b77c3cfac55 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:58.084574 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897eb3c5-8eca-4bac-94f2-3b77c3cfac55 None admin] RBAC: Adding query filter params (role.id=8abeff2cb87d4b7fa72b27b028005877, scope.project.id=fe863fcd75f1406ebbffc5016078f73e, user.id=992de6f86b42495dafe73cb2c393fb4e) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:59:58.085075 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897eb3c5-8eca-4bac-94f2-3b77c3cfac55 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:58.086691 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897eb3c5-8eca-4bac-94f2-3b77c3cfac55 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:58.089734 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc45bb56-e5f6-4b7c-9718-858e03c41655 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:58.093631 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 276/543] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Sep 9 11:59:57 2022] GET /identity/v3/role_assignments?user.id=992de6f86b42495dafe73cb2c393fb4e&scope.project.id=fe863fcd75f1406ebbffc5016078f73e&role.id=8abeff2cb87d4b7fa72b27b028005877 => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:58.213542 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:58.241586 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc45bb56-e5f6-4b7c-9718-858e03c41655 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:58.287947 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637f4d8a-cab6-4f19-a2bb-8d56d9644523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:58.288602 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637f4d8a-cab6-4f19-a2bb-8d56d9644523 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:58.288880 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637f4d8a-cab6-4f19-a2bb-8d56d9644523 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:58.289149 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637f4d8a-cab6-4f19-a2bb-8d56d9644523 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:58.310050 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637f4d8a-cab6-4f19-a2bb-8d56d9644523 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 11:59:58.315678 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-637f4d8a-cab6-4f19-a2bb-8d56d9644523 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 11:59:58.315678 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:59:58.315678 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:59:58.315678 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:59:58.315678 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:59:58.315678 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:59:58.315678 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:59:58.315678 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:59:58.315678 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:59:58.315678 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:59:58.315678 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:59:58.315678 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:59:58.315678 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 11:59:58.315678 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 11:59:58.315678 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 11:59:58.315678 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 11:59:58.316535 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 11:59:58.316535 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 11:59:58.316535 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 11:59:58.316535 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 11:59:58.316535 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:59:58.316535 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:59:58.316535 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:59:58.316535 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:59:58.316535 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:59:58.316535 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:59:58.316535 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:59:58.316535 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 11:59:58.316535 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:59:58.316535 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:59:58.316535 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:59:58.316535 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:59:58.318608 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:59:58.318608 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:59:58.318608 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:59:58.318608 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:59:58.318608 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 11:59:58.318608 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 11:59:58.318608 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 11:59:58.318608 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 11:59:58.318608 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 11:59:58.318608 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 11:59:58.318608 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 11:59:58.318608 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 11:59:58.318608 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 268/544] 10.222.0.49 () {56 vars in 1102 bytes} [Fri Sep 9 11:59:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:59:58.323309 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897eb3c5-8eca-4bac-94f2-3b77c3cfac55 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:58.452582 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897eb3c5-8eca-4bac-94f2-3b77c3cfac55 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:58.508221 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1bc18e-fe73-4c35-8364-c6263b29846e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:58.508841 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1bc18e-fe73-4c35-8364-c6263b29846e None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:58.509115 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1bc18e-fe73-4c35-8364-c6263b29846e None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:58.509391 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1bc18e-fe73-4c35-8364-c6263b29846e None admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:58.510055 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1bc18e-fe73-4c35-8364-c6263b29846e None admin] RBAC: Adding query filter params (name=service) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:59:58.510354 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1bc18e-fe73-4c35-8364-c6263b29846e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:58.511740 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1bc18e-fe73-4c35-8364-c6263b29846e None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:58.524044 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 277/545] 10.222.0.49 () {56 vars in 1103 bytes} [Fri Sep 9 11:59:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 11:59:58.529714 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637f4d8a-cab6-4f19-a2bb-8d56d9644523 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:58.674953 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637f4d8a-cab6-4f19-a2bb-8d56d9644523 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:58.728716 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb45335-2f9a-453c-aaed-506c8c0f004b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:58.729365 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb45335-2f9a-453c-aaed-506c8c0f004b None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:58.729649 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb45335-2f9a-453c-aaed-506c8c0f004b None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:58.729912 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb45335-2f9a-453c-aaed-506c8c0f004b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:58.736011 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:58.745136 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb45335-2f9a-453c-aaed-506c8c0f004b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:58.746605 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb45335-2f9a-453c-aaed-506c8c0f004b None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:58.762089 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-adb45335-2f9a-453c-aaed-506c8c0f004b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:59:58.762089 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:59:58.762089 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:59:58.762089 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:59:58.762089 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:59:58.762089 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:59:58.762089 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:59:58.762089 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:59:58.762089 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:59:58.762089 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:59:58.762089 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:59:58.762089 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:59:58.762089 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 11:59:58.762089 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 11:59:58.762089 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 11:59:58.762089 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 11:59:58.763040 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:59:58.763040 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:59:58.763040 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:59:58.763040 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:59:58.763040 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:59:58.763040 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:59:58.763040 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:59:58.763040 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 11:59:58.763040 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:59:58.763040 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:59:58.763040 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:59:58.763040 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:59:58.763040 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:59:58.763040 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:59:58.763040 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:59:58.763040 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:59:58.763040 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 11:59:58.763881 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 11:59:58.763881 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 11:59:58.763881 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 11:59:58.767014 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 269/546] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 11:59:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:59:58.769174 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1bc18e-fe73-4c35-8364-c6263b29846e None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:58.842042 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:58.914925 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1bc18e-fe73-4c35-8364-c6263b29846e None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:58.964553 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b285fac-5601-48a7-a8aa-10561c52b36d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:58.965173 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b285fac-5601-48a7-a8aa-10561c52b36d None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:58.965444 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b285fac-5601-48a7-a8aa-10561c52b36d None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:58.965709 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b285fac-5601-48a7-a8aa-10561c52b36d None admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:58.966409 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b285fac-5601-48a7-a8aa-10561c52b36d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:59:58.966708 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b285fac-5601-48a7-a8aa-10561c52b36d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:58.968090 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b285fac-5601-48a7-a8aa-10561c52b36d None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:58.989142 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 278/547] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 9 11:59:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:58.994848 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb45335-2f9a-453c-aaed-506c8c0f004b None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:59.147397 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb45335-2f9a-453c-aaed-506c8c0f004b None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:59.203858 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a6854f-4add-477a-8bb2-e0437c70e7b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:59.204721 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a6854f-4add-477a-8bb2-e0437c70e7b7 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:59.205147 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a6854f-4add-477a-8bb2-e0437c70e7b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:59.205575 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a6854f-4add-477a-8bb2-e0437c70e7b7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:59.217139 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a6854f-4add-477a-8bb2-e0437c70e7b7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:59.218802 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a6854f-4add-477a-8bb2-e0437c70e7b7 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:59.235348 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-b3a6854f-4add-477a-8bb2-e0437c70e7b7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 09 11:59:59.235348 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 11:59:59.235348 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 11:59:59.235348 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 11:59:59.235348 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 11:59:59.235348 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 11:59:59.235348 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 11:59:59.235348 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 11:59:59.235348 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 11:59:59.235348 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 11:59:59.235348 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 11:59:59.235348 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 11:59:59.235348 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 11:59:59.235348 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 11:59:59.235348 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 11:59:59.235348 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 11:59:59.236989 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 11:59:59.236989 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 11:59:59.236989 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 11:59:59.236989 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:59:59.236989 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 11:59:59.236989 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 11:59:59.236989 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 11:59:59.236989 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 11:59:59.236989 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 11:59:59.236989 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 11:59:59.236989 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 11:59:59.236989 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 11:59:59.236989 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 11:59:59.236989 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 11:59:59.236989 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 11:59:59.236989 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 11:59:59.236989 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 11:59:59.236989 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 11:59:59.238282 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 11:59:59.238282 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 11:59:59.238282 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 11:59:59.238282 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 11:59:59.238282 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 11:59:59.238282 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 11:59:59.238282 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 11:59:59.238282 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 11:59:59.238282 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 09 11:59:59.238282 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 11:59:59.243848 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 270/548] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Sep 9 11:59:58 2022] GET /identity/v3/users/swift => generated 83 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 11:59:59.247992 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b285fac-5601-48a7-a8aa-10561c52b36d None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:59.395352 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b285fac-5601-48a7-a8aa-10561c52b36d None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:59.401035 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:59.447804 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe00e15-8e57-4a02-b1dd-228507ff4359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:59.448412 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe00e15-8e57-4a02-b1dd-228507ff4359 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:59.448686 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe00e15-8e57-4a02-b1dd-228507ff4359 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:59.448953 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe00e15-8e57-4a02-b1dd-228507ff4359 None admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:59.449677 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe00e15-8e57-4a02-b1dd-228507ff4359 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 11:59:59.449976 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe00e15-8e57-4a02-b1dd-228507ff4359 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:59.451492 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe00e15-8e57-4a02-b1dd-228507ff4359 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:59.480565 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 279/549] 10.222.0.49 () {56 vars in 1135 bytes} [Fri Sep 9 11:59:59 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 11:59:59.486001 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a6854f-4add-477a-8bb2-e0437c70e7b7 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 11:59:59.536249 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0fb661-8078-4ae5-a11d-5f80c42097cf None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:59.536569 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0fb661-8078-4ae5-a11d-5f80c42097cf None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:59.536784 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0fb661-8078-4ae5-a11d-5f80c42097cf None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:59.538347 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 280/550] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 11:59:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 11:59:59.543162 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde5d4e0-b1d8-4996-9d78-5a97ae4b331b None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:59.543440 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde5d4e0-b1d8-4996-9d78-5a97ae4b331b None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:59.543709 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde5d4e0-b1d8-4996-9d78-5a97ae4b331b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:59.576334 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 11:59:59.578764 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-cde5d4e0-b1d8-4996-9d78-5a97ae4b331b None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:59:59.584641 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-cde5d4e0-b1d8-4996-9d78-5a97ae4b331b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:59.691505 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a6854f-4add-477a-8bb2-e0437c70e7b7 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 11:59:59.721046 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 281/551] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 11:59:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 11:59:59.750495 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dc59f4-0080-4be1-bec9-657b900ad6ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 11:59:59.751142 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dc59f4-0080-4be1-bec9-657b900ad6ab None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:59.751422 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dc59f4-0080-4be1-bec9-657b900ad6ab None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:59.751693 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dc59f4-0080-4be1-bec9-657b900ad6ab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:59.836157 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dc59f4-0080-4be1-bec9-657b900ad6ab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 11:59:59.837614 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dc59f4-0080-4be1-bec9-657b900ad6ab None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 11:59:59.841522 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0445528-62ff-4896-8090-ce491ac1e8ab None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:59.841804 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0445528-62ff-4896-8090-ce491ac1e8ab None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:59.842074 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0445528-62ff-4896-8090-ce491ac1e8ab None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:59.843718 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 282/552] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 11:59:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 11:59:59.859366 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f00378-043f-41b4-aff4-76cd8c3d2a38 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 11:59:59.859695 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f00378-043f-41b4-aff4-76cd8c3d2a38 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 11:59:59.859970 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f00378-043f-41b4-aff4-76cd8c3d2a38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 11:59:59.900858 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-a2f00378-043f-41b4-aff4-76cd8c3d2a38 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 11:59:59.902043 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-a2f00378-043f-41b4-aff4-76cd8c3d2a38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 11:59:59.917882 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:00.019603 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 271/553] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 9 11:59:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 534 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 12:00:00.020799 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dc59f4-0080-4be1-bec9-657b900ad6ab None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:00.022257 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 283/554] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 11:59:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 12:00:00.025865 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f00378-043f-41b4-aff4-76cd8c3d2a38 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:00.068547 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:00.197653 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dc59f4-0080-4be1-bec9-657b900ad6ab None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:00.223782 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f00378-043f-41b4-aff4-76cd8c3d2a38 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:00.281495 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa610c80-535d-4ea0-8924-7590b2619e90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:00.282053 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa610c80-535d-4ea0-8924-7590b2619e90 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:00.283287 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa610c80-535d-4ea0-8924-7590b2619e90 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:00.283623 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa610c80-535d-4ea0-8924-7590b2619e90 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:00.289112 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc355da-2d14-4cd9-b5ff-47a69e879386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:00.289752 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc355da-2d14-4cd9-b5ff-47a69e879386 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:00.290021 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc355da-2d14-4cd9-b5ff-47a69e879386 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:00.290282 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc355da-2d14-4cd9-b5ff-47a69e879386 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:00.308322 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa610c80-535d-4ea0-8924-7590b2619e90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:00:00.311808 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc355da-2d14-4cd9-b5ff-47a69e879386 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:00.313266 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc355da-2d14-4cd9-b5ff-47a69e879386 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:00.313451 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-fa610c80-535d-4ea0-8924-7590b2619e90 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 12:00:00.313451 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:00.313451 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:00.313451 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:00.313451 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:00.313451 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:00.313451 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:00.313451 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:00.313451 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:00.313451 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:00.313451 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:00.313451 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:00.313451 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:00:00.313451 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:00:00.313451 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:00:00.313451 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:00:00.314344 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:00:00.314344 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:00:00.314344 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:00:00.314344 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:00:00.314344 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:00.314344 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:00.314344 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:00.314344 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:00.314344 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:00.314344 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:00.314344 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:00.314344 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:00.314344 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:00.314344 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:00.314344 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:00.314344 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:00.315835 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:00.315835 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:00.315835 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:00.315835 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:00.315835 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:00:00.315835 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:00:00.315835 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:00:00.315835 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:00:00.315835 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:00:00.315835 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:00:00.315835 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 12:00:00.315835 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:00:00.315835 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 272/555] 10.222.0.49 () {56 vars in 1102 bytes} [Fri Sep 9 12:00:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:00.317137 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e0c701-3d46-493b-b2d5-0bb1ff60574b None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:00.317484 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e0c701-3d46-493b-b2d5-0bb1ff60574b None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:00.318642 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e0c701-3d46-493b-b2d5-0bb1ff60574b None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:00.319202 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 273/556] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:00:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:00:00.324354 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e0c701-3d46-493b-b2d5-0bb1ff60574b None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:00.335657 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:00.339381 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-ddc355da-2d14-4cd9-b5ff-47a69e879386 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:00:00.339381 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:00.339381 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:00.339381 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:00.339381 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:00.339381 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:00.339381 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:00.339381 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:00.339381 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:00.339381 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:00.339381 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:00.339381 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:00.339381 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:00:00.339381 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:00:00.339381 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:00:00.339381 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:00:00.341684 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:00.341684 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:00.341684 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:00.341684 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:00.341684 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:00.341684 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:00.341684 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:00.341684 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:00.341684 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:00.341684 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:00.341684 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:00.341684 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:00.341684 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:00.341684 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:00.341684 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:00.341684 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:00.341684 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:00:00.343855 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:00:00.343855 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:00:00.343855 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:00:00.343855 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 284/557] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 12:00:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:00.343855 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af5c476-22ea-4ad1-8a73-84a8e71968b0 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:00.343855 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af5c476-22ea-4ad1-8a73-84a8e71968b0 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:00.343855 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af5c476-22ea-4ad1-8a73-84a8e71968b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:00.360969 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-6af5c476-22ea-4ad1-8a73-84a8e71968b0 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:00:00.362143 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-6af5c476-22ea-4ad1-8a73-84a8e71968b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:00.437213 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e0c701-3d46-493b-b2d5-0bb1ff60574b None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:00.443110 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 285/558] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:00:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 12:00:00.443110 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af5c476-22ea-4ad1-8a73-84a8e71968b0 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:00.445661 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:00.486010 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e9531c-b058-466d-97e8-094671125a82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:00.486634 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e9531c-b058-466d-97e8-094671125a82 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:00.486925 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e9531c-b058-466d-97e8-094671125a82 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:00.487198 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e9531c-b058-466d-97e8-094671125a82 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:00.500924 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e9531c-b058-466d-97e8-094671125a82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:00:00.506383 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-96e9531c-b058-466d-97e8-094671125a82 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 12:00:00.506383 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:00.506383 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:00.506383 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:00.506383 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:00.506383 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:00.506383 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:00.506383 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:00.506383 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:00.506383 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:00.506383 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:00.506383 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:00.506383 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:00:00.506383 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:00:00.506383 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:00:00.506383 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:00:00.507851 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:00:00.507851 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:00:00.507851 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:00:00.507851 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:00:00.507851 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:00.507851 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:00.507851 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:00.507851 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:00.507851 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:00.507851 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:00.507851 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:00.507851 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:00.507851 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:00.507851 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:00.507851 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:00.507851 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:00.508671 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:00.508671 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:00.508671 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:00.508671 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:00.508671 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:00:00.508671 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:00:00.508671 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:00:00.508671 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:00:00.508671 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:00:00.508671 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:00:00.508671 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 12:00:00.508671 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:00:00.511240 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 274/559] 10.222.0.49 () {56 vars in 1102 bytes} [Fri Sep 9 12:00:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:00.512324 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e9531c-b058-466d-97e8-094671125a82 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:00.584790 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:00.588402 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af5c476-22ea-4ad1-8a73-84a8e71968b0 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:00.620320 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e9531c-b058-466d-97e8-094671125a82 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:00.643623 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451c22f0-2a0b-49aa-8302-1cbad37548a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:00.647214 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451c22f0-2a0b-49aa-8302-1cbad37548a4 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:00.647496 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451c22f0-2a0b-49aa-8302-1cbad37548a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:00.647765 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451c22f0-2a0b-49aa-8302-1cbad37548a4 None admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:00.648434 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451c22f0-2a0b-49aa-8302-1cbad37548a4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:00.649156 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451c22f0-2a0b-49aa-8302-1cbad37548a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:00.650532 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451c22f0-2a0b-49aa-8302-1cbad37548a4 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:00.660077 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 286/560] 10.222.0.49 () {56 vars in 1103 bytes} [Fri Sep 9 12:00:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:00.661533 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451c22f0-2a0b-49aa-8302-1cbad37548a4 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:00.668086 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb9ea6e-fc16-4c4f-94d8-4fc2da2e93b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:00.668977 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb9ea6e-fc16-4c4f-94d8-4fc2da2e93b6 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:00.669449 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb9ea6e-fc16-4c4f-94d8-4fc2da2e93b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:00.669914 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb9ea6e-fc16-4c4f-94d8-4fc2da2e93b6 None admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:00.671383 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb9ea6e-fc16-4c4f-94d8-4fc2da2e93b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:00.671885 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb9ea6e-fc16-4c4f-94d8-4fc2da2e93b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:00.673906 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb9ea6e-fc16-4c4f-94d8-4fc2da2e93b6 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:00.690674 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 275/561] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 9 12:00:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:00.692768 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb9ea6e-fc16-4c4f-94d8-4fc2da2e93b6 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:00.741112 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:00.781346 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451c22f0-2a0b-49aa-8302-1cbad37548a4 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:00.812306 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb9ea6e-fc16-4c4f-94d8-4fc2da2e93b6 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:00.823523 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45816c5-1825-4818-9ded-2519db9c8483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:00.824350 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45816c5-1825-4818-9ded-2519db9c8483 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:00.824783 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45816c5-1825-4818-9ded-2519db9c8483 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:00.825236 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45816c5-1825-4818-9ded-2519db9c8483 None admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:00.826047 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45816c5-1825-4818-9ded-2519db9c8483 None admin] RBAC: Adding query filter params (name=service) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:00.826499 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45816c5-1825-4818-9ded-2519db9c8483 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:00.828053 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45816c5-1825-4818-9ded-2519db9c8483 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:00.838004 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 287/562] 10.222.0.49 () {56 vars in 1103 bytes} [Fri Sep 9 12:00:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:00.839053 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45816c5-1825-4818-9ded-2519db9c8483 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:00.845079 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:00.854090 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f674c5-688d-4f06-9200-3943013f2542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:00.854731 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f674c5-688d-4f06-9200-3943013f2542 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:00.855044 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f674c5-688d-4f06-9200-3943013f2542 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:00.855317 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f674c5-688d-4f06-9200-3943013f2542 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:00.866414 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f674c5-688d-4f06-9200-3943013f2542 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:00.867930 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f674c5-688d-4f06-9200-3943013f2542 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:00.882296 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-32f674c5-688d-4f06-9200-3943013f2542 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:00:00.882296 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:00.882296 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:00.882296 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:00.882296 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:00.882296 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:00.882296 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:00.882296 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:00.882296 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:00.882296 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:00.882296 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:00.882296 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:00.882296 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:00:00.882296 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:00:00.882296 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:00:00.882296 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:00:00.883275 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:00.883275 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:00.883275 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:00.883275 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:00.883275 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:00.883275 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:00.883275 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:00.883275 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:00.883275 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:00.883275 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:00.883275 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:00.883275 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:00.883275 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:00.883275 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:00.883275 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:00.883275 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:00.883275 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:00:00.884154 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:00:00.884154 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:00:00.884154 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:00:00.884154 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 276/563] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 12:00:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:00.884154 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f674c5-688d-4f06-9200-3943013f2542 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:00.944707 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45816c5-1825-4818-9ded-2519db9c8483 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:00.970004 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:00.978375 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388448ba-ce61-4e38-8d15-f159069491f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:00.979029 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388448ba-ce61-4e38-8d15-f159069491f2 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:00.979305 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388448ba-ce61-4e38-8d15-f159069491f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:00.979573 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388448ba-ce61-4e38-8d15-f159069491f2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:00.986132 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388448ba-ce61-4e38-8d15-f159069491f2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:00.987597 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388448ba-ce61-4e38-8d15-f159069491f2 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:00.990007 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f674c5-688d-4f06-9200-3943013f2542 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:00.999220 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-388448ba-ce61-4e38-8d15-f159069491f2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:00:00.999220 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:00.999220 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:00.999220 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:00.999220 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:00.999220 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:00.999220 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:00.999220 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:00.999220 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:00.999220 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:00.999220 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:00.999220 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:00.999220 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 12:00:00.999220 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 12:00:00.999220 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 12:00:01.000092 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 12:00:01.000092 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:01.000092 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:01.000092 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:01.000092 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:01.000092 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:01.000092 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:01.000092 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:01.000092 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:01.000092 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:01.000092 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:01.000092 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:01.000092 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:01.000092 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:01.000092 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:01.000092 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:01.000092 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:01.000972 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 12:00:01.000972 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 12:00:01.000972 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 12:00:01.000972 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 12:00:01.000972 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 12:00:01.000972 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 12:00:01.000972 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:00:01.000972 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:00:01.000972 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 288/564] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 9 12:00:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:01.001505 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bef46d-e307-4a53-8be9-922e6cbc28ec None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:01.001696 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bef46d-e307-4a53-8be9-922e6cbc28ec None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:01.001969 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bef46d-e307-4a53-8be9-922e6cbc28ec None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:01.003209 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 289/565] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:00:01 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:00:01.005254 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bef46d-e307-4a53-8be9-922e6cbc28ec None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:01.026614 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f37e5c-93c5-416d-9baf-c3993a55b827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:01.027259 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f37e5c-93c5-416d-9baf-c3993a55b827 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:01.027530 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f37e5c-93c5-416d-9baf-c3993a55b827 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:01.027793 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f37e5c-93c5-416d-9baf-c3993a55b827 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:01.038713 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f37e5c-93c5-416d-9baf-c3993a55b827 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:00:01.043995 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-f8f37e5c-93c5-416d-9baf-c3993a55b827 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 12:00:01.043995 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:01.043995 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:01.043995 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:01.043995 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:01.043995 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:01.043995 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:01.043995 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:01.043995 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:01.043995 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:01.043995 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:01.043995 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:01.043995 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:00:01.043995 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:00:01.043995 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:00:01.043995 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:00:01.044905 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:00:01.044905 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:00:01.044905 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:00:01.044905 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:00:01.044905 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:01.044905 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:01.044905 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:01.044905 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:01.044905 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:01.044905 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:01.044905 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:01.044905 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:01.044905 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:01.044905 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:01.044905 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:01.044905 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:01.045726 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:01.045726 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:01.045726 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:01.045726 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:01.045726 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:00:01.045726 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:00:01.045726 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:00:01.045726 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:00:01.045726 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:00:01.045726 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:00:01.045726 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 12:00:01.045726 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:00:01.045726 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 277/566] 10.222.0.49 () {56 vars in 1102 bytes} [Fri Sep 9 12:00:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:01.046427 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f37e5c-93c5-416d-9baf-c3993a55b827 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:01.091085 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:01.110272 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bef46d-e307-4a53-8be9-922e6cbc28ec None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:01.143286 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623f4f3d-48bc-4786-b233-3ffa8208acbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:01.143904 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623f4f3d-48bc-4786-b233-3ffa8208acbd None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:01.144233 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623f4f3d-48bc-4786-b233-3ffa8208acbd None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:01.144503 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623f4f3d-48bc-4786-b233-3ffa8208acbd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:01.149977 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f37e5c-93c5-416d-9baf-c3993a55b827 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:01.155444 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623f4f3d-48bc-4786-b233-3ffa8208acbd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:01.156886 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623f4f3d-48bc-4786-b233-3ffa8208acbd None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:01.170831 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-623f4f3d-48bc-4786-b233-3ffa8208acbd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:00:01.170831 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:01.170831 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:01.170831 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:01.170831 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:01.170831 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:01.170831 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:01.170831 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:01.170831 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:01.170831 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:01.170831 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:01.170831 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:01.170831 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:00:01.170831 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:00:01.170831 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:00:01.170831 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:00:01.171780 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:01.171780 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:01.171780 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:01.171780 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:01.171780 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:01.171780 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:01.171780 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:01.171780 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:01.171780 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:01.171780 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:01.171780 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:01.171780 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:01.171780 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:01.171780 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:01.171780 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:01.171780 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:01.171780 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:00:01.172676 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:00:01.172676 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:00:01.172676 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:00:01.172676 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 290/567] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 12:00:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:01.172676 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623f4f3d-48bc-4786-b233-3ffa8208acbd None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:01.187620 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210afc91-cb40-41b1-95db-f44f0cb7ff80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:01.188229 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210afc91-cb40-41b1-95db-f44f0cb7ff80 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:01.188513 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210afc91-cb40-41b1-95db-f44f0cb7ff80 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:01.188782 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210afc91-cb40-41b1-95db-f44f0cb7ff80 None admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:01.189426 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210afc91-cb40-41b1-95db-f44f0cb7ff80 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:01.189995 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210afc91-cb40-41b1-95db-f44f0cb7ff80 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:01.191415 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210afc91-cb40-41b1-95db-f44f0cb7ff80 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:01.203643 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 278/568] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 9 12:00:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:01.204830 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cda52b-ef3b-4f27-9950-d1169d8932f2 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:01.205112 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cda52b-ef3b-4f27-9950-d1169d8932f2 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:01.205538 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cda52b-ef3b-4f27-9950-d1169d8932f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:01.216333 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:01.222681 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-c5cda52b-ef3b-4f27-9950-d1169d8932f2 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:00:01.224037 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-c5cda52b-ef3b-4f27-9950-d1169d8932f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:01.276217 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623f4f3d-48bc-4786-b233-3ffa8208acbd None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:01.289111 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 279/569] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:00:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 12:00:01.289947 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cda52b-ef3b-4f27-9950-d1169d8932f2 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:01.311956 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968d68ea-66dd-468a-a54b-85e044fe3a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:01.312568 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968d68ea-66dd-468a-a54b-85e044fe3a71 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:01.312851 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968d68ea-66dd-468a-a54b-85e044fe3a71 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:01.313122 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968d68ea-66dd-468a-a54b-85e044fe3a71 None admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:01.313819 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968d68ea-66dd-468a-a54b-85e044fe3a71 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:01.314124 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968d68ea-66dd-468a-a54b-85e044fe3a71 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:01.315853 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968d68ea-66dd-468a-a54b-85e044fe3a71 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:01.322245 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:01.334939 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 291/570] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Sep 9 12:00:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:01.335311 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968d68ea-66dd-468a-a54b-85e044fe3a71 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:01.407300 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cda52b-ef3b-4f27-9950-d1169d8932f2 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:01.443043 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d203819-efd0-46c0-9afd-facbcf6108ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:01.443279 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:01.443651 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d203819-efd0-46c0-9afd-facbcf6108ec None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:01.443930 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d203819-efd0-46c0-9afd-facbcf6108ec None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:01.444250 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d203819-efd0-46c0-9afd-facbcf6108ec None admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:01.444945 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d203819-efd0-46c0-9afd-facbcf6108ec None admin] RBAC: Adding query filter params (name=service) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:01.445160 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d203819-efd0-46c0-9afd-facbcf6108ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:01.446588 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d203819-efd0-46c0-9afd-facbcf6108ec None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:01.447221 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968d68ea-66dd-468a-a54b-85e044fe3a71 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:01.461374 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 280/571] 10.222.0.49 () {56 vars in 1103 bytes} [Fri Sep 9 12:00:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:01.461374 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d203819-efd0-46c0-9afd-facbcf6108ec None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:01.483613 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358534f3-1300-4e6d-88ce-4b131a22cf8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:01.484220 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358534f3-1300-4e6d-88ce-4b131a22cf8a None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:01.484752 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358534f3-1300-4e6d-88ce-4b131a22cf8a None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:01.484752 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358534f3-1300-4e6d-88ce-4b131a22cf8a None admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:01.485386 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358534f3-1300-4e6d-88ce-4b131a22cf8a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:01.485687 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358534f3-1300-4e6d-88ce-4b131a22cf8a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:01.487137 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358534f3-1300-4e6d-88ce-4b131a22cf8a None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:01.499333 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 292/572] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 9 12:00:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:01.500027 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358534f3-1300-4e6d-88ce-4b131a22cf8a None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:01.562894 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:01.572898 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d203819-efd0-46c0-9afd-facbcf6108ec None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:01.603273 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358534f3-1300-4e6d-88ce-4b131a22cf8a None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:01.605747 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bcbb0f-f751-422a-bbc2-1ff4e0205816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:01.606379 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bcbb0f-f751-422a-bbc2-1ff4e0205816 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:01.606657 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bcbb0f-f751-422a-bbc2-1ff4e0205816 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:01.606955 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bcbb0f-f751-422a-bbc2-1ff4e0205816 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:01.614259 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bcbb0f-f751-422a-bbc2-1ff4e0205816 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:01.615716 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bcbb0f-f751-422a-bbc2-1ff4e0205816 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:01.626611 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-23bcbb0f-f751-422a-bbc2-1ff4e0205816 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 12:00:01.626611 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:01.626611 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:01.626611 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:01.626611 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:01.626611 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:01.626611 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:01.626611 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:01.626611 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:01.626611 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:01.626611 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:01.626611 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:01.626611 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 12:00:01.626611 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 12:00:01.626611 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 12:00:01.626611 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 12:00:01.627606 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:01.627606 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:01.627606 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 12:00:01.627606 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:00:01.627606 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 12:00:01.627606 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:00:01.627606 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:01.627606 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:01.627606 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:01.627606 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:01.627606 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:01.627606 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:01.627606 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:01.627606 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:01.627606 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:01.627606 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:01.627606 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:01.627606 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:01.628792 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:01.628792 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:01.628792 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 12:00:01.628792 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 12:00:01.628792 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 12:00:01.628792 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 12:00:01.628792 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 12:00:01.628792 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 12:00:01.628792 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 12:00:01.628792 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:00:01.628792 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 281/573] 10.222.0.49 () {56 vars in 1093 bytes} [Fri Sep 9 12:00:01 2022] GET /identity/v3/users/nova => generated 82 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:01.628792 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bcbb0f-f751-422a-bbc2-1ff4e0205816 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:01.642677 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fec9e29-0997-4a97-a2b7-86cd98743998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:01.643316 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fec9e29-0997-4a97-a2b7-86cd98743998 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:01.643600 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fec9e29-0997-4a97-a2b7-86cd98743998 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:01.643869 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fec9e29-0997-4a97-a2b7-86cd98743998 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:01.644584 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fec9e29-0997-4a97-a2b7-86cd98743998 None admin] RBAC: Adding query filter params (role.id=8abeff2cb87d4b7fa72b27b028005877, scope.project.id=fe863fcd75f1406ebbffc5016078f73e, user.id=92e652ba827849b5a3fc1213d04a1b76) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:01.645592 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fec9e29-0997-4a97-a2b7-86cd98743998 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:01.647135 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fec9e29-0997-4a97-a2b7-86cd98743998 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:01.652486 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 293/574] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Sep 9 12:00:01 2022] GET /identity/v3/role_assignments?user.id=92e652ba827849b5a3fc1213d04a1b76&scope.project.id=fe863fcd75f1406ebbffc5016078f73e&role.id=8abeff2cb87d4b7fa72b27b028005877 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:01.653264 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fec9e29-0997-4a97-a2b7-86cd98743998 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:01.688876 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:01.744504 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bcbb0f-f751-422a-bbc2-1ff4e0205816 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:01.775730 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fec9e29-0997-4a97-a2b7-86cd98743998 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:01.783748 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49ce406-5303-4fda-9ee5-eda0c1d1a306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:01.784680 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49ce406-5303-4fda-9ee5-eda0c1d1a306 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:01.784952 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49ce406-5303-4fda-9ee5-eda0c1d1a306 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:01.785223 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49ce406-5303-4fda-9ee5-eda0c1d1a306 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:01.797260 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49ce406-5303-4fda-9ee5-eda0c1d1a306 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:01.798756 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49ce406-5303-4fda-9ee5-eda0c1d1a306 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:01.814870 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d569d4b6-4d40-4f7c-80b4-cdde4648eb21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:01.815386 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-c49ce406-5303-4fda-9ee5-eda0c1d1a306 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:00:01.815386 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:01.815386 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:01.815386 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:01.815386 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:01.815386 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:01.815386 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:01.815386 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:01.815386 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:01.815386 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:01.815386 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:01.815386 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:01.815386 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:00:01.815386 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:00:01.815386 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:00:01.815386 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:00:01.816326 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:01.816326 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:01.816326 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:01.816326 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:01.816326 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:01.816326 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:01.816326 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:01.816326 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:01.816326 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:01.816326 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:01.816326 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:01.816326 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:01.816326 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:01.816326 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:01.816326 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:01.816326 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:01.816326 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:00:01.817808 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:00:01.817808 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:00:01.817808 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:00:01.817990 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d569d4b6-4d40-4f7c-80b4-cdde4648eb21 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:01.817990 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d569d4b6-4d40-4f7c-80b4-cdde4648eb21 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:01.817990 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d569d4b6-4d40-4f7c-80b4-cdde4648eb21 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:01.818182 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 282/575] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 12:00:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:01.818182 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49ce406-5303-4fda-9ee5-eda0c1d1a306 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:01.823041 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:01.833256 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d569d4b6-4d40-4f7c-80b4-cdde4648eb21 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:01.835077 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d569d4b6-4d40-4f7c-80b4-cdde4648eb21 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:01.847608 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-d569d4b6-4d40-4f7c-80b4-cdde4648eb21 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 09 12:00:01.847608 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:01.847608 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:01.847608 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:01.847608 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:01.847608 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:01.847608 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:01.847608 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:01.847608 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:01.847608 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:01.847608 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:01.847608 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:01.847608 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 12:00:01.847608 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 12:00:01.847608 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 12:00:01.847608 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 12:00:01.849531 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:01.849531 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:01.849531 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 12:00:01.849531 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:00:01.849531 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 12:00:01.849531 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:00:01.849531 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:01.849531 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:01.849531 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:01.849531 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:01.849531 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:01.849531 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:01.849531 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:01.849531 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:01.849531 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:01.849531 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:01.849531 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:01.849531 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:01.850485 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:01.850485 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:01.850485 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 12:00:01.850485 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 12:00:01.850485 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 12:00:01.850485 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 12:00:01.850485 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 12:00:01.850485 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 12:00:01.850485 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 09 12:00:01.850485 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:00:01.850485 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 294/576] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 9 12:00:01 2022] GET /identity/v3/users/glance => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:01.851318 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d569d4b6-4d40-4f7c-80b4-cdde4648eb21 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:01.935575 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49ce406-5303-4fda-9ee5-eda0c1d1a306 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:01.944770 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:01.962567 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d569d4b6-4d40-4f7c-80b4-cdde4648eb21 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:01.974746 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84fa6d7-8a7c-4696-a086-9f4e2a150403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:01.975662 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84fa6d7-8a7c-4696-a086-9f4e2a150403 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:01.975939 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84fa6d7-8a7c-4696-a086-9f4e2a150403 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:01.976211 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84fa6d7-8a7c-4696-a086-9f4e2a150403 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:01.986913 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84fa6d7-8a7c-4696-a086-9f4e2a150403 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:00:01.992341 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-b84fa6d7-8a7c-4696-a086-9f4e2a150403 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 12:00:01.992341 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:01.992341 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:01.992341 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:01.992341 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:01.992341 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:01.992341 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:01.992341 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:01.992341 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:01.992341 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:01.992341 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:01.992341 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:01.992341 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:00:01.992341 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:00:01.992341 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:00:01.992341 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:00:01.993261 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:00:01.993261 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:00:01.993261 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:00:01.993261 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:00:01.993261 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:01.993261 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:01.993261 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:01.993261 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:01.993261 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:01.993261 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:01.993261 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:01.993261 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:01.993261 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:01.993261 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:01.993261 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:01.993261 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:01.994120 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:01.994120 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:01.994120 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:01.994120 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:01.994120 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:00:01.994120 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:00:01.994120 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:00:01.994120 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:00:01.994120 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:00:01.994120 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:00:01.994120 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 12:00:01.994120 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:00:01.998271 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 283/577] 10.222.0.49 () {56 vars in 1102 bytes} [Fri Sep 9 12:00:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:01.999049 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84fa6d7-8a7c-4696-a086-9f4e2a150403 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:02.000899 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2d9e93-4309-4e93-942b-32d02f2a64fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:02.001679 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2d9e93-4309-4e93-942b-32d02f2a64fe None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:02.001960 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2d9e93-4309-4e93-942b-32d02f2a64fe None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:02.002390 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2d9e93-4309-4e93-942b-32d02f2a64fe None admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:02.003161 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2d9e93-4309-4e93-942b-32d02f2a64fe None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:02.003468 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2d9e93-4309-4e93-942b-32d02f2a64fe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:02.004981 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2d9e93-4309-4e93-942b-32d02f2a64fe None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:02.029638 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 295/578] 10.222.0.49 () {56 vars in 1133 bytes} [Fri Sep 9 12:00:01 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:02.029638 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2d9e93-4309-4e93-942b-32d02f2a64fe None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:02.068348 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:02.112192 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84fa6d7-8a7c-4696-a086-9f4e2a150403 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:02.139902 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2d9e93-4309-4e93-942b-32d02f2a64fe None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:02.151867 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b55c2ea-0e82-4675-8d28-a17b6a96741e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:02.152475 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b55c2ea-0e82-4675-8d28-a17b6a96741e None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:02.152759 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b55c2ea-0e82-4675-8d28-a17b6a96741e None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:02.153027 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b55c2ea-0e82-4675-8d28-a17b6a96741e None admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:02.153667 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b55c2ea-0e82-4675-8d28-a17b6a96741e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:02.153970 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b55c2ea-0e82-4675-8d28-a17b6a96741e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:02.155388 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b55c2ea-0e82-4675-8d28-a17b6a96741e None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:02.167697 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 284/579] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 9 12:00:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:02.168515 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b55c2ea-0e82-4675-8d28-a17b6a96741e None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:02.186277 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058f1eb3-f3b4-40d4-b4fa-1697eb4b6eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:02.187055 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058f1eb3-f3b4-40d4-b4fa-1697eb4b6eb9 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:02.187311 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058f1eb3-f3b4-40d4-b4fa-1697eb4b6eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:02.187585 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058f1eb3-f3b4-40d4-b4fa-1697eb4b6eb9 None admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:02.188310 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058f1eb3-f3b4-40d4-b4fa-1697eb4b6eb9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:02.188611 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058f1eb3-f3b4-40d4-b4fa-1697eb4b6eb9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:02.190303 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058f1eb3-f3b4-40d4-b4fa-1697eb4b6eb9 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:02.204483 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:02.213439 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 296/580] 10.222.0.49 () {56 vars in 1137 bytes} [Fri Sep 9 12:00:02 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:02.214319 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058f1eb3-f3b4-40d4-b4fa-1697eb4b6eb9 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:02.282333 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b55c2ea-0e82-4675-8d28-a17b6a96741e None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:02.307866 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:02.321167 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9bd1e1-6277-4fe5-aebc-1f64659a4fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:02.321773 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9bd1e1-6277-4fe5-aebc-1f64659a4fcd None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:02.322050 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9bd1e1-6277-4fe5-aebc-1f64659a4fcd None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:02.322320 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9bd1e1-6277-4fe5-aebc-1f64659a4fcd None admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:02.322590 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058f1eb3-f3b4-40d4-b4fa-1697eb4b6eb9 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:02.323040 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9bd1e1-6277-4fe5-aebc-1f64659a4fcd None admin] RBAC: Adding query filter params (name=service) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:02.323298 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9bd1e1-6277-4fe5-aebc-1f64659a4fcd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:02.324792 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9bd1e1-6277-4fe5-aebc-1f64659a4fcd None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:02.338268 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 285/581] 10.222.0.49 () {56 vars in 1103 bytes} [Fri Sep 9 12:00:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:02.338268 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9bd1e1-6277-4fe5-aebc-1f64659a4fcd None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:02.364701 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b9260c-4750-4b15-9369-ba8f09b4ce10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:02.365695 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b9260c-4750-4b15-9369-ba8f09b4ce10 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:02.366393 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b9260c-4750-4b15-9369-ba8f09b4ce10 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:02.366775 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b9260c-4750-4b15-9369-ba8f09b4ce10 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:02.380502 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b9260c-4750-4b15-9369-ba8f09b4ce10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:02.382712 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b9260c-4750-4b15-9369-ba8f09b4ce10 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:02.401175 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-29b9260c-4750-4b15-9369-ba8f09b4ce10 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:00:02.401175 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:02.401175 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:02.401175 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:02.401175 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:02.401175 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:02.401175 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:02.401175 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:02.401175 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:02.401175 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:02.401175 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:02.401175 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:02.401175 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:00:02.401175 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:00:02.401175 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:00:02.401175 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:00:02.402356 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:02.402356 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:02.402356 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:02.402356 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:02.402356 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:02.402356 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:02.402356 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:02.402356 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:02.402356 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:02.402356 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:02.402356 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:02.402356 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:02.402356 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:02.402356 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:02.402356 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:02.402356 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:02.402356 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:00:02.403504 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:00:02.403504 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:00:02.403504 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:00:02.409253 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 297/582] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 12:00:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:02.409253 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b9260c-4750-4b15-9369-ba8f09b4ce10 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:02.451254 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9bd1e1-6277-4fe5-aebc-1f64659a4fcd None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:02.471619 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:02.488908 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a826132-0779-4425-ae3c-16c37b71e892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:02.489630 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a826132-0779-4425-ae3c-16c37b71e892 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:02.490721 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a826132-0779-4425-ae3c-16c37b71e892 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:02.491030 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a826132-0779-4425-ae3c-16c37b71e892 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:02.499994 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a826132-0779-4425-ae3c-16c37b71e892 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:02.501550 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a826132-0779-4425-ae3c-16c37b71e892 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:02.513653 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-9a826132-0779-4425-ae3c-16c37b71e892 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 12:00:02.513653 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:02.513653 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:02.513653 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:02.513653 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:02.513653 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:02.513653 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:02.513653 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:02.513653 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:02.513653 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:02.513653 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:02.513653 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:02.513653 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 12:00:02.513653 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 12:00:02.513653 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 12:00:02.513653 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 12:00:02.514815 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:02.514815 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:02.514815 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 12:00:02.514815 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:00:02.514815 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 12:00:02.514815 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:00:02.514815 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:02.514815 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:02.514815 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:02.514815 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:02.514815 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:02.514815 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:02.514815 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:02.514815 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:02.514815 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:02.514815 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:02.514815 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:02.514815 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:02.516031 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:02.516031 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:02.516031 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 12:00:02.516031 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 12:00:02.516031 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 12:00:02.516031 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 12:00:02.516031 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 12:00:02.516031 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 12:00:02.516031 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 12:00:02.516031 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:00:02.516834 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 286/583] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 9 12:00:02 2022] GET /identity/v3/users/cinder => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:02.517649 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a826132-0779-4425-ae3c-16c37b71e892 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:02.531134 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b9260c-4750-4b15-9369-ba8f09b4ce10 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:02.569302 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:02.573278 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0462c2-65be-4cca-8a15-3c39367d7448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:02.573912 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0462c2-65be-4cca-8a15-3c39367d7448 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:02.574186 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0462c2-65be-4cca-8a15-3c39367d7448 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:02.574450 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0462c2-65be-4cca-8a15-3c39367d7448 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:02.587423 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0462c2-65be-4cca-8a15-3c39367d7448 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:02.588904 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0462c2-65be-4cca-8a15-3c39367d7448 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:02.606868 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-dc0462c2-65be-4cca-8a15-3c39367d7448 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:00:02.606868 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:02.606868 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:02.606868 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:02.606868 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:02.606868 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:02.606868 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:02.606868 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:02.606868 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:02.606868 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:02.606868 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:02.606868 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:02.606868 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:00:02.606868 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:00:02.606868 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:00:02.606868 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:00:02.607860 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:02.607860 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:02.607860 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:02.607860 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:02.607860 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:02.607860 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:02.607860 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:02.607860 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:02.607860 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:02.607860 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:02.607860 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:02.607860 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:02.607860 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:02.607860 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:02.607860 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:02.607860 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:02.607860 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:00:02.608738 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:00:02.608738 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:00:02.608738 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:00:02.608738 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 298/584] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 12:00:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:02.609278 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0462c2-65be-4cca-8a15-3c39367d7448 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:02.634632 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a826132-0779-4425-ae3c-16c37b71e892 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:02.673734 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b591f925-bb04-484c-bd88-7d1e65cdba6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:02.674436 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b591f925-bb04-484c-bd88-7d1e65cdba6f None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:02.674710 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b591f925-bb04-484c-bd88-7d1e65cdba6f None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:02.675007 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b591f925-bb04-484c-bd88-7d1e65cdba6f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:02.686303 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b591f925-bb04-484c-bd88-7d1e65cdba6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:02.687921 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b591f925-bb04-484c-bd88-7d1e65cdba6f None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:02.703236 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-b591f925-bb04-484c-bd88-7d1e65cdba6f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:00:02.703236 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:02.703236 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:02.703236 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:02.703236 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:02.703236 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:02.703236 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:02.703236 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:02.703236 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:02.703236 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:02.703236 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:02.703236 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:02.703236 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:00:02.703236 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:00:02.703236 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:00:02.703236 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:00:02.704360 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:02.704360 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:02.704360 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:02.704360 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:02.704360 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:02.704360 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:02.704360 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:02.704360 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:02.704360 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:02.704360 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:02.704360 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:02.704360 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:02.704360 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:02.704360 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:02.704360 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:02.704360 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:02.704360 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:00:02.710411 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:00:02.710411 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:00:02.710411 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:00:02.710411 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 287/585] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 12:00:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:02.710411 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b591f925-bb04-484c-bd88-7d1e65cdba6f None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:02.729518 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0462c2-65be-4cca-8a15-3c39367d7448 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:02.731920 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:02.764094 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061ef1b3-112e-42af-be33-17c666bf9cd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:02.764878 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061ef1b3-112e-42af-be33-17c666bf9cd3 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:02.765265 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061ef1b3-112e-42af-be33-17c666bf9cd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:02.765638 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061ef1b3-112e-42af-be33-17c666bf9cd3 None admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:02.766288 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061ef1b3-112e-42af-be33-17c666bf9cd3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:02.766619 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061ef1b3-112e-42af-be33-17c666bf9cd3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:02.767986 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061ef1b3-112e-42af-be33-17c666bf9cd3 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:02.782948 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 299/586] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 9 12:00:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:02.782948 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061ef1b3-112e-42af-be33-17c666bf9cd3 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:02.817546 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:02.821499 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b591f925-bb04-484c-bd88-7d1e65cdba6f None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:02.857158 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bcaaa3-c0e3-4a43-b476-9deb66184d77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:02.857763 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bcaaa3-c0e3-4a43-b476-9deb66184d77 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:02.858039 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bcaaa3-c0e3-4a43-b476-9deb66184d77 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:02.858308 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bcaaa3-c0e3-4a43-b476-9deb66184d77 None admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:02.859060 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bcaaa3-c0e3-4a43-b476-9deb66184d77 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:02.859361 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bcaaa3-c0e3-4a43-b476-9deb66184d77 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:02.860803 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bcaaa3-c0e3-4a43-b476-9deb66184d77 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:02.885761 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061ef1b3-112e-42af-be33-17c666bf9cd3 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:02.904028 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 288/587] 10.222.0.49 () {56 vars in 1137 bytes} [Fri Sep 9 12:00:02 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:02.904028 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bcaaa3-c0e3-4a43-b476-9deb66184d77 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:02.928997 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be6cb7b-b534-4773-9062-a597fded367c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:02.929699 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be6cb7b-b534-4773-9062-a597fded367c None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:02.930045 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be6cb7b-b534-4773-9062-a597fded367c None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:02.930383 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be6cb7b-b534-4773-9062-a597fded367c None admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:02.931056 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be6cb7b-b534-4773-9062-a597fded367c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:02.931353 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be6cb7b-b534-4773-9062-a597fded367c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:02.932711 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be6cb7b-b534-4773-9062-a597fded367c None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:02.949388 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 300/588] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 9 12:00:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:02.949388 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be6cb7b-b534-4773-9062-a597fded367c None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:02.977084 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:03.009949 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bcaaa3-c0e3-4a43-b476-9deb66184d77 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:03.050128 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e43d662-7007-4232-bf58-5b57a6eee6de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:03.050947 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e43d662-7007-4232-bf58-5b57a6eee6de None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:03.051742 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e43d662-7007-4232-bf58-5b57a6eee6de None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:03.052013 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e43d662-7007-4232-bf58-5b57a6eee6de None admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:03.052991 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e43d662-7007-4232-bf58-5b57a6eee6de None admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:03.053565 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e43d662-7007-4232-bf58-5b57a6eee6de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:03.055146 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e43d662-7007-4232-bf58-5b57a6eee6de None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:03.058179 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be6cb7b-b534-4773-9062-a597fded367c None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:03.069457 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 289/589] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 9 12:00:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:03.070808 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e43d662-7007-4232-bf58-5b57a6eee6de None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:03.092545 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:03.095448 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc666cd4-aeee-49ef-9aa4-a22c4cb9e24a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:03.096141 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc666cd4-aeee-49ef-9aa4-a22c4cb9e24a None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:03.096427 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc666cd4-aeee-49ef-9aa4-a22c4cb9e24a None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:03.096717 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc666cd4-aeee-49ef-9aa4-a22c4cb9e24a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:03.104079 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc666cd4-aeee-49ef-9aa4-a22c4cb9e24a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:03.105521 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc666cd4-aeee-49ef-9aa4-a22c4cb9e24a None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:03.117615 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-dc666cd4-aeee-49ef-9aa4-a22c4cb9e24a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:00:03.117615 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:03.117615 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:03.117615 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:03.117615 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:03.117615 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:03.117615 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:03.117615 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:03.117615 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:03.117615 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:03.117615 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:03.117615 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:03.117615 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 12:00:03.117615 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 12:00:03.117615 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 12:00:03.118676 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 12:00:03.118676 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:03.118676 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:03.118676 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:03.118676 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:03.118676 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:03.118676 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:03.118676 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:03.118676 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:03.118676 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:03.118676 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:03.118676 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:03.118676 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:03.118676 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:03.118676 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:03.118676 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:03.118676 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:03.119634 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 12:00:03.119634 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 12:00:03.119634 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 12:00:03.119634 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 12:00:03.119634 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 12:00:03.119634 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 12:00:03.119634 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:00:03.119634 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:00:03.119634 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 301/590] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 9 12:00:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:03.120471 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc666cd4-aeee-49ef-9aa4-a22c4cb9e24a None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:03.186280 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e43d662-7007-4232-bf58-5b57a6eee6de None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:03.211413 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:03.222874 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164731db-7308-4ace-be33-9beb3cb95134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:03.224062 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164731db-7308-4ace-be33-9beb3cb95134 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:03.224339 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164731db-7308-4ace-be33-9beb3cb95134 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:03.224606 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164731db-7308-4ace-be33-9beb3cb95134 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:03.228692 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc666cd4-aeee-49ef-9aa4-a22c4cb9e24a None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:03.235158 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164731db-7308-4ace-be33-9beb3cb95134 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:03.236600 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164731db-7308-4ace-be33-9beb3cb95134 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:03.251700 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-164731db-7308-4ace-be33-9beb3cb95134 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:00:03.251700 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:03.251700 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:03.251700 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:03.251700 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:03.251700 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:03.251700 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:03.251700 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:03.251700 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:03.251700 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:03.251700 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:03.251700 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:03.251700 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:00:03.251700 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:00:03.251700 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:00:03.251700 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:00:03.252659 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:03.252659 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:03.252659 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:03.252659 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:03.252659 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:03.252659 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:03.252659 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:03.252659 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:03.252659 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:03.252659 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:03.252659 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:03.252659 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:03.252659 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:03.252659 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:03.252659 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:03.252659 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:03.252659 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:00:03.253587 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:00:03.253587 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:00:03.253587 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:00:03.253587 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 290/591] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 12:00:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:03.253822 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164731db-7308-4ace-be33-9beb3cb95134 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:03.270584 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071b2683-cefe-471f-adab-9f48dde234c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:03.271296 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071b2683-cefe-471f-adab-9f48dde234c7 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:03.271574 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071b2683-cefe-471f-adab-9f48dde234c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:03.271843 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071b2683-cefe-471f-adab-9f48dde234c7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:03.279164 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071b2683-cefe-471f-adab-9f48dde234c7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:03.280830 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071b2683-cefe-471f-adab-9f48dde234c7 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:03.292794 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-071b2683-cefe-471f-adab-9f48dde234c7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:00:03.292794 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:03.292794 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:03.292794 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:03.292794 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:03.292794 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:03.292794 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:03.292794 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:03.292794 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:03.292794 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:03.292794 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:03.292794 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:03.292794 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 12:00:03.292794 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 12:00:03.292794 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 12:00:03.293696 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 12:00:03.293696 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:03.293696 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:03.293696 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:03.293696 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:03.293696 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:03.293696 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:03.293696 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:03.293696 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:03.293696 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:03.293696 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:03.293696 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:03.293696 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:03.293696 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:03.293696 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:03.293696 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:03.293696 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:03.294587 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 12:00:03.294587 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 12:00:03.294587 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 12:00:03.294587 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 12:00:03.294587 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 12:00:03.294587 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 12:00:03.294587 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:00:03.294587 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:00:03.294587 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 302/592] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 9 12:00:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:03.295110 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071b2683-cefe-471f-adab-9f48dde234c7 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:03.326883 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:03.366066 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164731db-7308-4ace-be33-9beb3cb95134 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:03.403876 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071b2683-cefe-471f-adab-9f48dde234c7 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:03.404404 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc09f11-cc5c-4bb2-83a3-6be420d1033f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:03.404667 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc09f11-cc5c-4bb2-83a3-6be420d1033f None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:03.404942 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc09f11-cc5c-4bb2-83a3-6be420d1033f None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:03.405205 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc09f11-cc5c-4bb2-83a3-6be420d1033f None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:03.413114 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc09f11-cc5c-4bb2-83a3-6be420d1033f None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:03.414567 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc09f11-cc5c-4bb2-83a3-6be420d1033f None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:03.427383 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-1dc09f11-cc5c-4bb2-83a3-6be420d1033f None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 09 12:00:03.427383 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:03.427383 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:03.427383 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:03.427383 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:03.427383 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:03.427383 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:03.427383 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:03.427383 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:03.427383 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:03.427383 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:03.427383 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:03.427383 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 12:00:03.427383 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 12:00:03.427383 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 12:00:03.427383 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 12:00:03.428502 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:03.428502 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:03.428502 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 12:00:03.428502 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:00:03.428502 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 12:00:03.428502 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:00:03.428502 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:03.428502 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:03.428502 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:03.428502 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:03.428502 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:03.428502 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:03.428502 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:03.428502 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:03.428502 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:03.428502 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:03.428502 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:03.428502 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:03.434291 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:03.434291 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:03.434291 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 12:00:03.434291 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 12:00:03.434291 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 12:00:03.434291 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 12:00:03.434291 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 12:00:03.434291 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 12:00:03.434291 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 09 12:00:03.434291 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:00:03.434291 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 291/593] 10.222.0.49 () {56 vars in 1102 bytes} [Fri Sep 9 12:00:03 2022] GET /identity/v3/users/neutron => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:03.434291 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc09f11-cc5c-4bb2-83a3-6be420d1033f None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:03.447745 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bf6311-3cce-4a3d-b68a-2c468e0164bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:03.448441 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bf6311-3cce-4a3d-b68a-2c468e0164bd None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:03.448802 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bf6311-3cce-4a3d-b68a-2c468e0164bd None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:03.449191 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bf6311-3cce-4a3d-b68a-2c468e0164bd None admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:03.449955 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bf6311-3cce-4a3d-b68a-2c468e0164bd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:03.450337 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bf6311-3cce-4a3d-b68a-2c468e0164bd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:03.451854 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bf6311-3cce-4a3d-b68a-2c468e0164bd None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:03.468760 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 303/594] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Sep 9 12:00:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:03.468760 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bf6311-3cce-4a3d-b68a-2c468e0164bd None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:03.472812 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:03.540816 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc09f11-cc5c-4bb2-83a3-6be420d1033f None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:03.572131 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bf6311-3cce-4a3d-b68a-2c468e0164bd None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:03.572847 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:03.585468 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd5450f-feef-47df-84c4-04c3cdaa5dc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:03.586312 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd5450f-feef-47df-84c4-04c3cdaa5dc0 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:03.586684 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd5450f-feef-47df-84c4-04c3cdaa5dc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:03.587078 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd5450f-feef-47df-84c4-04c3cdaa5dc0 None admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:03.587842 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd5450f-feef-47df-84c4-04c3cdaa5dc0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:03.588235 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd5450f-feef-47df-84c4-04c3cdaa5dc0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:03.589705 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd5450f-feef-47df-84c4-04c3cdaa5dc0 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:03.608080 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 292/595] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 9 12:00:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:03.608080 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd5450f-feef-47df-84c4-04c3cdaa5dc0 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:03.614528 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e0fef1-c045-4a88-83f5-e9b6ca18e61e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:03.615151 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e0fef1-c045-4a88-83f5-e9b6ca18e61e None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:03.615428 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e0fef1-c045-4a88-83f5-e9b6ca18e61e None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:03.615693 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e0fef1-c045-4a88-83f5-e9b6ca18e61e None admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:03.616374 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e0fef1-c045-4a88-83f5-e9b6ca18e61e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:03.616672 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e0fef1-c045-4a88-83f5-e9b6ca18e61e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:03.618091 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e0fef1-c045-4a88-83f5-e9b6ca18e61e None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:03.637131 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 304/596] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Sep 9 12:00:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:03.637131 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e0fef1-c045-4a88-83f5-e9b6ca18e61e None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:03.702473 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:03.728412 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd5450f-feef-47df-84c4-04c3cdaa5dc0 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:03.748269 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e0fef1-c045-4a88-83f5-e9b6ca18e61e None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:03.764013 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a74bcc-1163-412f-b85a-4bfcf67559c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:03.764621 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a74bcc-1163-412f-b85a-4bfcf67559c9 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:03.764896 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a74bcc-1163-412f-b85a-4bfcf67559c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:03.765168 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a74bcc-1163-412f-b85a-4bfcf67559c9 None admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:03.765887 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a74bcc-1163-412f-b85a-4bfcf67559c9 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:03.766187 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a74bcc-1163-412f-b85a-4bfcf67559c9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:03.767649 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a74bcc-1163-412f-b85a-4bfcf67559c9 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:03.783801 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5094b57a-eff1-453a-8b1c-83640ca09ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:03.784458 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5094b57a-eff1-453a-8b1c-83640ca09ce3 None admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:03.784737 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5094b57a-eff1-453a-8b1c-83640ca09ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:03.785008 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5094b57a-eff1-453a-8b1c-83640ca09ce3 None admin] PATH_INFO: `/v3/projects/fe863fcd75f1406ebbffc5016078f73e/users/70bac169d5cd43e1926f310344ac11fa/roles/8abeff2cb87d4b7fa72b27b028005877` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:03.791330 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 293/597] 10.222.0.49 () {56 vars in 1139 bytes} [Fri Sep 9 12:00:03 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:03.791330 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a74bcc-1163-412f-b85a-4bfcf67559c9 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:03.806397 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5094b57a-eff1-453a-8b1c-83640ca09ce3 None admin] RBAC: Authorizing `identity:create_grant(project_id=fe863fcd75f1406ebbffc5016078f73e, user_id=70bac169d5cd43e1926f310344ac11fa, role_id=8abeff2cb87d4b7fa72b27b028005877)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:03.807874 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5094b57a-eff1-453a-8b1c-83640ca09ce3 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:03.831096 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 305/598] 10.222.0.49 () {58 vars in 1439 bytes} [Fri Sep 9 12:00:03 2022] PUT /identity/v3/projects/fe863fcd75f1406ebbffc5016078f73e/users/70bac169d5cd43e1926f310344ac11fa/roles/8abeff2cb87d4b7fa72b27b028005877 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:00:03.831096 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5094b57a-eff1-453a-8b1c-83640ca09ce3 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:03.854859 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:03.916452 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a74bcc-1163-412f-b85a-4bfcf67559c9 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:03.942575 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5094b57a-eff1-453a-8b1c-83640ca09ce3 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:03.955667 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b3e685-1b38-48c4-8859-7a000a5e454f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:03.956370 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b3e685-1b38-48c4-8859-7a000a5e454f None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:03.956774 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b3e685-1b38-48c4-8859-7a000a5e454f None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:03.957079 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b3e685-1b38-48c4-8859-7a000a5e454f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:03.957742 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:03.966407 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b3e685-1b38-48c4-8859-7a000a5e454f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:03.967972 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b3e685-1b38-48c4-8859-7a000a5e454f None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:03.981150 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-71b3e685-1b38-48c4-8859-7a000a5e454f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:00:03.981150 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:03.981150 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:03.981150 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:03.981150 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:03.981150 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:03.981150 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:03.981150 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:03.981150 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:03.981150 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:03.981150 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:03.981150 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:03.981150 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 12:00:03.981150 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 12:00:03.981150 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 12:00:03.982287 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 12:00:03.982287 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:03.982287 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:03.982287 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:03.982287 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:03.982287 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:03.982287 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:03.982287 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:03.982287 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:03.982287 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:03.982287 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:03.982287 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:03.982287 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:03.982287 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:03.982287 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:03.982287 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:03.982287 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:03.983365 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 12:00:03.983365 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 12:00:03.983365 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 12:00:03.983365 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 12:00:03.983365 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 12:00:03.983365 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 12:00:03.983365 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:00:03.983365 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:00:03.984056 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 294/599] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 9 12:00:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:03.987871 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b3e685-1b38-48c4-8859-7a000a5e454f None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:03.992993 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79fc58e-0be4-4f27-9fb8-cfe69091758f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:03.993680 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79fc58e-0be4-4f27-9fb8-cfe69091758f None admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:03.993957 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79fc58e-0be4-4f27-9fb8-cfe69091758f None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:03.994225 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79fc58e-0be4-4f27-9fb8-cfe69091758f None admin] PATH_INFO: `/v3/projects/fe863fcd75f1406ebbffc5016078f73e/users/37c4e14aa8e04d3e85a406b9363bca2a/roles/8abeff2cb87d4b7fa72b27b028005877` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:04.013119 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79fc58e-0be4-4f27-9fb8-cfe69091758f None admin] RBAC: Authorizing `identity:create_grant(project_id=fe863fcd75f1406ebbffc5016078f73e, user_id=37c4e14aa8e04d3e85a406b9363bca2a, role_id=8abeff2cb87d4b7fa72b27b028005877)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:04.014574 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79fc58e-0be4-4f27-9fb8-cfe69091758f None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:04.027861 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 306/600] 10.222.0.49 () {58 vars in 1439 bytes} [Fri Sep 9 12:00:03 2022] PUT /identity/v3/projects/fe863fcd75f1406ebbffc5016078f73e/users/37c4e14aa8e04d3e85a406b9363bca2a/roles/8abeff2cb87d4b7fa72b27b028005877 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:00:04.028954 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e90c14-ab09-46e2-af59-ac480a7f285b None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:04.029253 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e90c14-ab09-46e2-af59-ac480a7f285b None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:04.029525 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e90c14-ab09-46e2-af59-ac480a7f285b None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:04.031030 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 307/601] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:00:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:00:04.031543 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e90c14-ab09-46e2-af59-ac480a7f285b None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:04.135227 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b3e685-1b38-48c4-8859-7a000a5e454f None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:04.142964 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:04.164634 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e90c14-ab09-46e2-af59-ac480a7f285b None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:04.176732 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f835bb51-f161-4201-8bfd-4cc1e8d5ac5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:04.177305 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f835bb51-f161-4201-8bfd-4cc1e8d5ac5e None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:04.177583 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f835bb51-f161-4201-8bfd-4cc1e8d5ac5e None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:04.177851 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f835bb51-f161-4201-8bfd-4cc1e8d5ac5e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:04.190302 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f835bb51-f161-4201-8bfd-4cc1e8d5ac5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:04.193370 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f835bb51-f161-4201-8bfd-4cc1e8d5ac5e None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:04.210706 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-f835bb51-f161-4201-8bfd-4cc1e8d5ac5e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:00:04.210706 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:04.210706 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:04.210706 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:04.210706 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:04.210706 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:04.210706 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:04.210706 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:04.210706 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:04.210706 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:04.210706 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:04.210706 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:04.210706 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:00:04.210706 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:00:04.210706 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:00:04.210706 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:00:04.211718 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:04.211718 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:04.211718 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:04.211718 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:04.211718 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:04.211718 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:04.211718 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:04.211718 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:04.211718 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:04.211718 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:04.211718 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:04.211718 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:04.211718 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:04.211718 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:04.211718 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:04.211718 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:04.211718 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:00:04.213022 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:00:04.213022 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:00:04.213022 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:00:04.213022 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 295/602] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 12:00:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:04.213424 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eb7057-54e5-44c9-b765-6fa4019cf6fd None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:04.213630 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eb7057-54e5-44c9-b765-6fa4019cf6fd None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:04.213906 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eb7057-54e5-44c9-b765-6fa4019cf6fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:04.215472 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c074f5-ee66-41bb-b664-d4f12b95c299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:04.216384 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c074f5-ee66-41bb-b664-d4f12b95c299 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:04.216886 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c074f5-ee66-41bb-b664-d4f12b95c299 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:04.217367 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c074f5-ee66-41bb-b664-d4f12b95c299 None admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:04.218258 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c074f5-ee66-41bb-b664-d4f12b95c299 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:04.218760 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c074f5-ee66-41bb-b664-d4f12b95c299 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:04.220495 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c074f5-ee66-41bb-b664-d4f12b95c299 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:04.234013 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 308/603] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Sep 9 12:00:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:04.234829 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c074f5-ee66-41bb-b664-d4f12b95c299 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:04.241003 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-29eb7057-54e5-44c9-b765-6fa4019cf6fd None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:00:04.242223 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-29eb7057-54e5-44c9-b765-6fa4019cf6fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:04.269275 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:04.312307 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 296/604] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:00:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 12:00:04.313374 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eb7057-54e5-44c9-b765-6fa4019cf6fd None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:04.356492 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c074f5-ee66-41bb-b664-d4f12b95c299 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:04.409472 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:04.420687 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c270a92-831a-4575-ac5e-7f8628511d37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:04.421997 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c270a92-831a-4575-ac5e-7f8628511d37 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:04.422499 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c270a92-831a-4575-ac5e-7f8628511d37 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:04.423000 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c270a92-831a-4575-ac5e-7f8628511d37 None admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:04.423553 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eb7057-54e5-44c9-b765-6fa4019cf6fd None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:04.424071 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c270a92-831a-4575-ac5e-7f8628511d37 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:04.424588 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c270a92-831a-4575-ac5e-7f8628511d37 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:04.427169 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c270a92-831a-4575-ac5e-7f8628511d37 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:04.440743 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 309/605] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 9 12:00:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:04.455016 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c270a92-831a-4575-ac5e-7f8628511d37 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:04.471320 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a21f21-ad62-4be7-a81e-37d7e5df173c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:04.471320 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a21f21-ad62-4be7-a81e-37d7e5df173c None admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:04.471320 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a21f21-ad62-4be7-a81e-37d7e5df173c None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:04.471320 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a21f21-ad62-4be7-a81e-37d7e5df173c None admin] PATH_INFO: `/v3/projects/fe863fcd75f1406ebbffc5016078f73e/users/535c258100df40b58a5ee44b2c21d621/roles/8abeff2cb87d4b7fa72b27b028005877` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:04.491029 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a21f21-ad62-4be7-a81e-37d7e5df173c None admin] RBAC: Authorizing `identity:create_grant(project_id=fe863fcd75f1406ebbffc5016078f73e, user_id=535c258100df40b58a5ee44b2c21d621, role_id=8abeff2cb87d4b7fa72b27b028005877)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:04.492504 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a21f21-ad62-4be7-a81e-37d7e5df173c None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:04.530387 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 297/606] 10.222.0.49 () {58 vars in 1439 bytes} [Fri Sep 9 12:00:04 2022] PUT /identity/v3/projects/fe863fcd75f1406ebbffc5016078f73e/users/535c258100df40b58a5ee44b2c21d621/roles/8abeff2cb87d4b7fa72b27b028005877 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:00:04.588768 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:04.607286 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c270a92-831a-4575-ac5e-7f8628511d37 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:04.631316 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a21f21-ad62-4be7-a81e-37d7e5df173c None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:04.672327 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43772ed-a720-4513-bac3-97afbd77b786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:04.674680 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43772ed-a720-4513-bac3-97afbd77b786 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:04.676575 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43772ed-a720-4513-bac3-97afbd77b786 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:04.677092 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43772ed-a720-4513-bac3-97afbd77b786 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:04.691354 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43772ed-a720-4513-bac3-97afbd77b786 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:04.694794 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43772ed-a720-4513-bac3-97afbd77b786 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:04.718185 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-c43772ed-a720-4513-bac3-97afbd77b786 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:00:04.718185 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:04.718185 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:04.718185 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:04.718185 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:04.718185 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:04.718185 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:04.718185 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:04.718185 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:04.718185 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:04.718185 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:04.718185 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:04.718185 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 12:00:04.718185 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 12:00:04.718185 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 12:00:04.719167 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 12:00:04.719167 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:04.719167 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:04.719167 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:04.719167 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:04.719167 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:04.719167 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:04.719167 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:04.719167 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:04.719167 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:04.719167 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:04.719167 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:04.719167 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:04.719167 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:04.719167 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:04.719167 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:04.719167 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:04.720094 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 12:00:04.720094 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 12:00:04.720094 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 12:00:04.720094 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 12:00:04.720094 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 12:00:04.720094 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 12:00:04.720094 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:00:04.720094 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:00:04.732041 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 310/607] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 9 12:00:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:04.740698 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43772ed-a720-4513-bac3-97afbd77b786 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:04.821157 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a21f21-ad62-4be7-a81e-37d7e5df173c None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:04.858380 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7666c477-0266-4e7a-8903-67de87520f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:04.859404 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7666c477-0266-4e7a-8903-67de87520f92 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:04.860037 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7666c477-0266-4e7a-8903-67de87520f92 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:04.860550 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7666c477-0266-4e7a-8903-67de87520f92 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:04.870238 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:04.875404 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7666c477-0266-4e7a-8903-67de87520f92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:00:04.882380 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43772ed-a720-4513-bac3-97afbd77b786 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:04.882999 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-7666c477-0266-4e7a-8903-67de87520f92 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 12:00:04.882999 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:04.882999 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:04.882999 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:04.882999 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:04.882999 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:04.882999 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:04.882999 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:04.882999 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:04.882999 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:04.882999 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:04.882999 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:04.882999 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:00:04.882999 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:00:04.882999 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:00:04.882999 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:00:04.883885 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:00:04.883885 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:00:04.883885 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:00:04.883885 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:00:04.883885 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:04.883885 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:04.883885 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:04.883885 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:04.883885 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:04.883885 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:04.883885 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:04.883885 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:04.883885 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:04.883885 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:04.883885 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:04.883885 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:04.884808 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:04.884808 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:04.884808 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:04.884808 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:04.884808 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:00:04.884808 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:00:04.884808 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:00:04.884808 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:00:04.884808 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:00:04.884808 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:00:04.884808 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 12:00:04.884808 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:00:04.887527 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 298/608] 10.222.0.49 () {56 vars in 1102 bytes} [Fri Sep 9 12:00:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 12:00:04.893152 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7666c477-0266-4e7a-8903-67de87520f92 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:04.920691 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed00e0a-b83f-45df-a4be-4fbead724477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:04.921308 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed00e0a-b83f-45df-a4be-4fbead724477 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:04.921590 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed00e0a-b83f-45df-a4be-4fbead724477 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:04.921866 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed00e0a-b83f-45df-a4be-4fbead724477 None admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:04.922791 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed00e0a-b83f-45df-a4be-4fbead724477 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:04.923074 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed00e0a-b83f-45df-a4be-4fbead724477 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:04.925122 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed00e0a-b83f-45df-a4be-4fbead724477 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:04.942935 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 311/609] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Sep 9 12:00:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:00:04.947776 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed00e0a-b83f-45df-a4be-4fbead724477 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:04.952565 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:05.017585 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7666c477-0266-4e7a-8903-67de87520f92 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:05.056331 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb676b2-2389-4ab0-8220-403163b2dca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:05.057336 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb676b2-2389-4ab0-8220-403163b2dca8 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:05.057814 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb676b2-2389-4ab0-8220-403163b2dca8 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:05.058284 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb676b2-2389-4ab0-8220-403163b2dca8 None admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:05.059166 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb676b2-2389-4ab0-8220-403163b2dca8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:05.059669 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb676b2-2389-4ab0-8220-403163b2dca8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:05.061271 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb676b2-2389-4ab0-8220-403163b2dca8 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:05.071359 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 299/610] 10.222.0.49 () {56 vars in 1103 bytes} [Fri Sep 9 12:00:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:05.077347 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb676b2-2389-4ab0-8220-403163b2dca8 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:05.159396 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed00e0a-b83f-45df-a4be-4fbead724477 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:05.199766 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:05.232432 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790fd88e-ca21-46fd-a992-32a9b2de3194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:05.233569 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790fd88e-ca21-46fd-a992-32a9b2de3194 None admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:05.234049 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790fd88e-ca21-46fd-a992-32a9b2de3194 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:05.234558 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790fd88e-ca21-46fd-a992-32a9b2de3194 None admin] PATH_INFO: `/v3/projects/fe863fcd75f1406ebbffc5016078f73e/users/992de6f86b42495dafe73cb2c393fb4e/roles/8abeff2cb87d4b7fa72b27b028005877` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:05.265749 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790fd88e-ca21-46fd-a992-32a9b2de3194 None admin] RBAC: Authorizing `identity:create_grant(project_id=fe863fcd75f1406ebbffc5016078f73e, user_id=992de6f86b42495dafe73cb2c393fb4e, role_id=8abeff2cb87d4b7fa72b27b028005877)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:05.268563 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790fd88e-ca21-46fd-a992-32a9b2de3194 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:05.283448 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb676b2-2389-4ab0-8220-403163b2dca8 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:05.304202 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 312/611] 10.222.0.49 () {58 vars in 1439 bytes} [Fri Sep 9 12:00:04 2022] PUT /identity/v3/projects/fe863fcd75f1406ebbffc5016078f73e/users/992de6f86b42495dafe73cb2c393fb4e/roles/8abeff2cb87d4b7fa72b27b028005877 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:00:05.332532 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c97023-1396-4137-90c0-46bc483c133e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:05.333656 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c97023-1396-4137-90c0-46bc483c133e None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:05.334165 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c97023-1396-4137-90c0-46bc483c133e None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:05.334681 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c97023-1396-4137-90c0-46bc483c133e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:05.355561 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c97023-1396-4137-90c0-46bc483c133e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:05.357465 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c97023-1396-4137-90c0-46bc483c133e None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:05.376338 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-77c97023-1396-4137-90c0-46bc483c133e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:00:05.376338 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:05.376338 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:05.376338 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:05.376338 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:05.376338 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:05.376338 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:05.376338 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:05.376338 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:05.376338 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:05.376338 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:05.376338 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:05.376338 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:00:05.376338 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:00:05.376338 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:00:05.376338 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:00:05.377395 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:05.377395 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:05.377395 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:05.377395 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:05.377395 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:05.377395 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:05.377395 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:05.377395 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:05.377395 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:05.377395 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:05.377395 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:05.377395 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:05.377395 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:05.377395 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:05.377395 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:05.377395 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:05.377395 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:00:05.378296 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:00:05.378296 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:00:05.378296 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:00:05.381415 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 300/612] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 12:00:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:05.386696 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790fd88e-ca21-46fd-a992-32a9b2de3194 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:05.454987 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:05.516037 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790fd88e-ca21-46fd-a992-32a9b2de3194 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:05.567435 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e697d254-aeef-415e-a259-ca975c4d886d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:05.568502 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e697d254-aeef-415e-a259-ca975c4d886d None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:05.569023 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e697d254-aeef-415e-a259-ca975c4d886d None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:05.569509 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e697d254-aeef-415e-a259-ca975c4d886d None admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:05.570410 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e697d254-aeef-415e-a259-ca975c4d886d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:05.570942 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e697d254-aeef-415e-a259-ca975c4d886d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:05.572551 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e697d254-aeef-415e-a259-ca975c4d886d None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:05.590992 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 313/613] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 9 12:00:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:05.596118 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c97023-1396-4137-90c0-46bc483c133e None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:05.714667 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c97023-1396-4137-90c0-46bc483c133e None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:05.730084 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:05.756920 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c70e24c-2b52-4080-b6fb-72d6f3b27e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:05.757899 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c70e24c-2b52-4080-b6fb-72d6f3b27e10 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:05.758381 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c70e24c-2b52-4080-b6fb-72d6f3b27e10 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:05.758882 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c70e24c-2b52-4080-b6fb-72d6f3b27e10 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:05.766975 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c70e24c-2b52-4080-b6fb-72d6f3b27e10 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:05.768662 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c70e24c-2b52-4080-b6fb-72d6f3b27e10 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:05.788847 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-0c70e24c-2b52-4080-b6fb-72d6f3b27e10 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 09 12:00:05.788847 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:05.788847 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:05.788847 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:05.788847 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:05.788847 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:05.788847 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:05.788847 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:05.788847 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:05.788847 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:05.788847 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:05.788847 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:05.788847 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 12:00:05.788847 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 12:00:05.788847 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 12:00:05.788847 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 12:00:05.789865 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:05.789865 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:05.789865 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 12:00:05.789865 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:00:05.789865 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 12:00:05.789865 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:00:05.789865 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:05.789865 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:05.789865 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:05.789865 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:05.789865 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:05.789865 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:05.789865 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:05.789865 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:05.789865 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:05.789865 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:05.789865 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:05.789865 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:05.790821 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:05.790821 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:05.790821 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 12:00:05.790821 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 12:00:05.790821 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 12:00:05.790821 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 12:00:05.790821 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 12:00:05.790821 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 12:00:05.790821 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 09 12:00:05.790821 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:00:05.793058 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 301/614] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Sep 9 12:00:05 2022] GET /identity/v3/users/swift => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:05.803983 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e697d254-aeef-415e-a259-ca975c4d886d None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:05.963676 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e697d254-aeef-415e-a259-ca975c4d886d None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:05.991107 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:06.020713 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f9dfcb-99da-46b6-9b65-848a4cb2c371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:06.021325 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f9dfcb-99da-46b6-9b65-848a4cb2c371 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:06.021600 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f9dfcb-99da-46b6-9b65-848a4cb2c371 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:06.021869 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f9dfcb-99da-46b6-9b65-848a4cb2c371 None admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:06.022746 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f9dfcb-99da-46b6-9b65-848a4cb2c371 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:06.023071 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f9dfcb-99da-46b6-9b65-848a4cb2c371 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:06.024482 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f9dfcb-99da-46b6-9b65-848a4cb2c371 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:06.055119 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 314/615] 10.222.0.49 () {56 vars in 1135 bytes} [Fri Sep 9 12:00:05 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:06.061477 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c70e24c-2b52-4080-b6fb-72d6f3b27e10 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:06.211305 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c70e24c-2b52-4080-b6fb-72d6f3b27e10 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:06.259378 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26d87de-f8b8-42eb-9170-b0135b33cf00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:06.260416 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26d87de-f8b8-42eb-9170-b0135b33cf00 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:06.260911 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26d87de-f8b8-42eb-9170-b0135b33cf00 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:06.261436 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26d87de-f8b8-42eb-9170-b0135b33cf00 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:06.282579 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26d87de-f8b8-42eb-9170-b0135b33cf00 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:06.284393 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26d87de-f8b8-42eb-9170-b0135b33cf00 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:06.307322 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-b26d87de-f8b8-42eb-9170-b0135b33cf00 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:00:06.307322 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:06.307322 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:06.307322 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:06.307322 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:06.307322 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:06.307322 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:06.307322 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:06.307322 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:06.307322 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:06.307322 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:06.307322 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:06.307322 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:00:06.307322 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:00:06.307322 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:00:06.307322 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:00:06.308378 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:06.308378 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:06.308378 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:06.308378 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:06.308378 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:06.308378 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:06.308378 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:06.308378 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:06.308378 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:06.308378 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:06.308378 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:06.308378 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:06.308378 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:06.308378 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:06.308378 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:06.308378 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:06.308378 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:00:06.309325 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:00:06.309325 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:00:06.309325 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:00:06.311147 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 302/616] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 12:00:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:06.317753 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f9dfcb-99da-46b6-9b65-848a4cb2c371 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:06.453913 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8c8f66-3e6e-4d2f-9cc8-9fb05ce714b5 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:06.454585 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8c8f66-3e6e-4d2f-9cc8-9fb05ce714b5 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:06.455134 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8c8f66-3e6e-4d2f-9cc8-9fb05ce714b5 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:06.457675 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 303/617] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:00:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:00:06.463394 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e49995-b0ca-494a-bdb9-97ab026123e6 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:06.464026 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e49995-b0ca-494a-bdb9-97ab026123e6 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:06.464407 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e49995-b0ca-494a-bdb9-97ab026123e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:06.485464 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f9dfcb-99da-46b6-9b65-848a4cb2c371 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:06.495999 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-c1e49995-b0ca-494a-bdb9-97ab026123e6 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:00:06.499543 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-c1e49995-b0ca-494a-bdb9-97ab026123e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:06.534792 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:06.549314 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da87b87-cb0a-4955-8d97-b338e018987d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:06.551727 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da87b87-cb0a-4955-8d97-b338e018987d None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:06.552341 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da87b87-cb0a-4955-8d97-b338e018987d None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:06.553362 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da87b87-cb0a-4955-8d97-b338e018987d None admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:06.554334 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da87b87-cb0a-4955-8d97-b338e018987d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:06.555444 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da87b87-cb0a-4955-8d97-b338e018987d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:06.557132 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da87b87-cb0a-4955-8d97-b338e018987d None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:06.583296 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 315/618] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 9 12:00:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:06.586552 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da87b87-cb0a-4955-8d97-b338e018987d None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:06.614638 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 304/619] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:00:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 12:00:06.616630 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae95fcfb-2da2-4a1e-a523-10280f7e0bf8 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:06.617100 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae95fcfb-2da2-4a1e-a523-10280f7e0bf8 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:06.617551 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae95fcfb-2da2-4a1e-a523-10280f7e0bf8 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:06.619204 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 305/620] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:00:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:00:06.637440 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aa9381-a09b-48ec-8737-a00b8168d035 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:06.638306 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aa9381-a09b-48ec-8737-a00b8168d035 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:06.638792 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aa9381-a09b-48ec-8737-a00b8168d035 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:06.647149 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:06.679060 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-c0aa9381-a09b-48ec-8737-a00b8168d035 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:00:06.680246 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-c0aa9381-a09b-48ec-8737-a00b8168d035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:06.779346 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da87b87-cb0a-4955-8d97-b338e018987d None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:06.782321 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 306/621] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:00:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 12:00:06.829625 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069cafdf-d8ad-4904-964c-fb239aa49a7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:06.830169 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069cafdf-d8ad-4904-964c-fb239aa49a7b None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:06.830441 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069cafdf-d8ad-4904-964c-fb239aa49a7b None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:06.830705 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069cafdf-d8ad-4904-964c-fb239aa49a7b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:06.840321 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069cafdf-d8ad-4904-964c-fb239aa49a7b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:06.841815 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069cafdf-d8ad-4904-964c-fb239aa49a7b None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:06.854813 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-069cafdf-d8ad-4904-964c-fb239aa49a7b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:00:06.854813 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:06.854813 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:06.854813 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:06.854813 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:06.854813 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:06.854813 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:06.854813 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:06.854813 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:06.854813 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:06.854813 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:06.854813 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:06.854813 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 12:00:06.854813 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 12:00:06.854813 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 12:00:06.855734 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 12:00:06.855734 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:06.855734 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:06.855734 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:06.855734 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:06.855734 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:06.855734 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:06.855734 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:06.855734 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:06.855734 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:06.855734 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:06.855734 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:06.855734 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:06.855734 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:06.855734 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:06.855734 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:06.855734 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:06.856596 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 12:00:06.856596 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 12:00:06.856596 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 12:00:06.856596 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 12:00:06.856596 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 12:00:06.856596 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 12:00:06.856596 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:00:06.856596 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:00:06.863022 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 316/622] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 9 12:00:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 12:00:06.867981 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aa9381-a09b-48ec-8737-a00b8168d035 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:06.949824 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069cafdf-d8ad-4904-964c-fb239aa49a7b None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:06.969928 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:06.979553 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aa9381-a09b-48ec-8737-a00b8168d035 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:07.015200 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ec06af-b913-4531-b685-f0b5a4e8a9ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:07.015809 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ec06af-b913-4531-b685-f0b5a4e8a9ac None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:07.016084 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ec06af-b913-4531-b685-f0b5a4e8a9ac None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:07.016356 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ec06af-b913-4531-b685-f0b5a4e8a9ac None admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:07.017037 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ec06af-b913-4531-b685-f0b5a4e8a9ac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:07.017337 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ec06af-b913-4531-b685-f0b5a4e8a9ac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:07.018856 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ec06af-b913-4531-b685-f0b5a4e8a9ac None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:07.031992 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 307/623] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Sep 9 12:00:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:07.039448 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ec06af-b913-4531-b685-f0b5a4e8a9ac None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:07.125243 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:07.146435 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069cafdf-d8ad-4904-964c-fb239aa49a7b None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:07.192009 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be754f7-cae0-4e66-aee5-f8cdc46c948c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:07.192652 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be754f7-cae0-4e66-aee5-f8cdc46c948c None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:07.192931 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be754f7-cae0-4e66-aee5-f8cdc46c948c None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:07.193234 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be754f7-cae0-4e66-aee5-f8cdc46c948c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:07.204739 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be754f7-cae0-4e66-aee5-f8cdc46c948c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:00:07.210448 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-3be754f7-cae0-4e66-aee5-f8cdc46c948c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 12:00:07.210448 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:07.210448 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:07.210448 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:07.210448 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:07.210448 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:07.210448 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:07.210448 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:07.210448 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:07.210448 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:07.210448 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:07.210448 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:07.210448 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:00:07.210448 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:00:07.210448 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:00:07.210448 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:00:07.211921 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:00:07.211921 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:00:07.211921 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:00:07.211921 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:00:07.211921 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:07.211921 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:07.211921 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:07.211921 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:07.211921 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:07.211921 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:07.211921 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:07.211921 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:07.211921 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:07.211921 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:07.211921 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:07.211921 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:07.212955 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:07.212955 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:07.212955 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:07.212955 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:07.212955 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:00:07.212955 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:00:07.212955 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:00:07.212955 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:00:07.212955 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:00:07.212955 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:00:07.212955 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 12:00:07.212955 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:00:07.216481 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 317/624] 10.222.0.49 () {56 vars in 1102 bytes} [Fri Sep 9 12:00:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:07.217343 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be754f7-cae0-4e66-aee5-f8cdc46c948c None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:07.220386 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ec06af-b913-4531-b685-f0b5a4e8a9ac None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:07.269144 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6f4718-c9a6-4850-8013-0872f3b5787b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:07.271018 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6f4718-c9a6-4850-8013-0872f3b5787b None admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:07.271719 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6f4718-c9a6-4850-8013-0872f3b5787b None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:07.272138 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6f4718-c9a6-4850-8013-0872f3b5787b None admin] PATH_INFO: `/v3/projects/fe863fcd75f1406ebbffc5016078f73e/users/92e652ba827849b5a3fc1213d04a1b76/roles/8abeff2cb87d4b7fa72b27b028005877` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:07.292955 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6f4718-c9a6-4850-8013-0872f3b5787b None admin] RBAC: Authorizing `identity:create_grant(project_id=fe863fcd75f1406ebbffc5016078f73e, user_id=92e652ba827849b5a3fc1213d04a1b76, role_id=8abeff2cb87d4b7fa72b27b028005877)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:07.295508 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6f4718-c9a6-4850-8013-0872f3b5787b None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:07.297873 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:07.315287 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 308/625] 10.222.0.49 () {58 vars in 1439 bytes} [Fri Sep 9 12:00:07 2022] PUT /identity/v3/projects/fe863fcd75f1406ebbffc5016078f73e/users/92e652ba827849b5a3fc1213d04a1b76/roles/8abeff2cb87d4b7fa72b27b028005877 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:00:07.316915 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6f4718-c9a6-4850-8013-0872f3b5787b None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:07.327991 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be754f7-cae0-4e66-aee5-f8cdc46c948c None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:07.370608 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578cce53-0702-4ed3-af94-5574e8c9b76e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:07.371445 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578cce53-0702-4ed3-af94-5574e8c9b76e None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:07.371730 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578cce53-0702-4ed3-af94-5574e8c9b76e None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:07.372059 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578cce53-0702-4ed3-af94-5574e8c9b76e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:07.384169 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578cce53-0702-4ed3-af94-5574e8c9b76e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:00:07.390891 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-578cce53-0702-4ed3-af94-5574e8c9b76e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 12:00:07.390891 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:07.390891 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:07.390891 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:07.390891 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:07.390891 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:07.390891 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:07.390891 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:07.390891 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:07.390891 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:07.390891 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:07.390891 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:07.390891 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:00:07.390891 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:00:07.390891 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:00:07.390891 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:00:07.392142 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:00:07.392142 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:00:07.392142 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:00:07.392142 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:00:07.392142 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:07.392142 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:07.392142 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:07.392142 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:07.392142 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:07.392142 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:07.392142 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:07.392142 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:07.392142 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:07.392142 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:07.392142 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:07.392142 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:07.393172 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:07.393172 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:07.393172 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:07.393172 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:07.393172 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:00:07.393172 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:00:07.393172 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:00:07.393172 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:00:07.393172 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:00:07.393172 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:00:07.393172 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 12:00:07.393172 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:00:07.396115 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 318/626] 10.222.0.49 () {56 vars in 1102 bytes} [Fri Sep 9 12:00:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:07.397764 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c311e4d-fe2e-4190-a331-5c4aa3992f67 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:07.398193 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c311e4d-fe2e-4190-a331-5c4aa3992f67 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:07.398607 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c311e4d-fe2e-4190-a331-5c4aa3992f67 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:07.400477 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 319/627] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:00:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:00:07.402740 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c311e4d-fe2e-4190-a331-5c4aa3992f67 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:07.424002 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:07.454530 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6f4718-c9a6-4850-8013-0872f3b5787b None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:07.502879 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c07afbd-4c59-40fc-92d2-7db06182c150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:07.504021 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c07afbd-4c59-40fc-92d2-7db06182c150 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:07.504863 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c07afbd-4c59-40fc-92d2-7db06182c150 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:07.505343 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c07afbd-4c59-40fc-92d2-7db06182c150 None admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:07.506892 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c07afbd-4c59-40fc-92d2-7db06182c150 None admin] RBAC: Adding query filter params (name=service) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:07.507736 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c07afbd-4c59-40fc-92d2-7db06182c150 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:07.509653 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c07afbd-4c59-40fc-92d2-7db06182c150 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:07.522579 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 309/628] 10.222.0.49 () {56 vars in 1103 bytes} [Fri Sep 9 12:00:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:07.524276 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01a924b-cdac-400d-9083-71f63dbe5537 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:07.524802 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01a924b-cdac-400d-9083-71f63dbe5537 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:07.525276 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01a924b-cdac-400d-9083-71f63dbe5537 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:07.544597 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-f01a924b-cdac-400d-9083-71f63dbe5537 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:00:07.545789 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-f01a924b-cdac-400d-9083-71f63dbe5537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:07.550454 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c311e4d-fe2e-4190-a331-5c4aa3992f67 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:07.596325 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a723a563-e35f-44c0-9624-75d676b413f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:07.597224 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a723a563-e35f-44c0-9624-75d676b413f5 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:07.597502 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a723a563-e35f-44c0-9624-75d676b413f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:07.598332 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a723a563-e35f-44c0-9624-75d676b413f5 None admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:07.599360 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a723a563-e35f-44c0-9624-75d676b413f5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:07.599483 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a723a563-e35f-44c0-9624-75d676b413f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:07.600811 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a723a563-e35f-44c0-9624-75d676b413f5 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:07.608771 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:07.611108 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 320/629] 10.222.0.49 () {56 vars in 1103 bytes} [Fri Sep 9 12:00:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:07.612219 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a723a563-e35f-44c0-9624-75d676b413f5 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:07.629670 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 310/630] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:00:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 12:00:07.631273 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01a924b-cdac-400d-9083-71f63dbe5537 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:07.747170 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01a924b-cdac-400d-9083-71f63dbe5537 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:07.787916 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd517bae-dc8c-4c96-ba5a-ba1064f2cf73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:07.789300 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:07.790584 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd517bae-dc8c-4c96-ba5a-ba1064f2cf73 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:07.791085 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd517bae-dc8c-4c96-ba5a-ba1064f2cf73 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:07.791535 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd517bae-dc8c-4c96-ba5a-ba1064f2cf73 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:07.805255 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd517bae-dc8c-4c96-ba5a-ba1064f2cf73 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:07.806923 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd517bae-dc8c-4c96-ba5a-ba1064f2cf73 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:07.824976 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-dd517bae-dc8c-4c96-ba5a-ba1064f2cf73 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:00:07.824976 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:07.824976 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:07.824976 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:07.824976 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:07.824976 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:07.824976 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:07.824976 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:07.824976 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:07.824976 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:07.824976 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:07.824976 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:07.824976 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:00:07.824976 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:00:07.824976 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:00:07.824976 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:00:07.826066 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:07.826066 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:07.826066 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:07.826066 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:07.826066 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:07.826066 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:07.826066 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:07.826066 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:07.826066 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:07.826066 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:07.826066 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:07.826066 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:07.826066 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:07.826066 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:07.826066 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:07.826066 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:07.826066 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:00:07.826977 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:00:07.826977 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:00:07.826977 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:00:07.826977 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 311/631] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 12:00:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:07.833525 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd517bae-dc8c-4c96-ba5a-ba1064f2cf73 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:07.879677 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:07.899827 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a723a563-e35f-44c0-9624-75d676b413f5 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:07.939864 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0cf03b-04b1-4fed-a397-86a7bda18b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:07.940663 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0cf03b-04b1-4fed-a397-86a7bda18b92 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:07.941008 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0cf03b-04b1-4fed-a397-86a7bda18b92 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:07.941437 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0cf03b-04b1-4fed-a397-86a7bda18b92 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:07.954714 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0cf03b-04b1-4fed-a397-86a7bda18b92 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:07.956800 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0cf03b-04b1-4fed-a397-86a7bda18b92 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:07.974294 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-8c0cf03b-04b1-4fed-a397-86a7bda18b92 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:00:07.974294 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:07.974294 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:07.974294 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:07.974294 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:07.974294 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:07.974294 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:07.974294 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:07.974294 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:07.974294 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:07.974294 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:07.974294 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:07.974294 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:00:07.974294 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:00:07.974294 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:00:07.974294 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:00:07.975366 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:07.975366 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:07.975366 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:07.975366 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:07.975366 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:07.975366 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:07.975366 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:07.975366 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:07.975366 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:07.975366 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:07.975366 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:07.975366 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:07.975366 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:07.975366 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:07.975366 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:07.975366 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:07.975366 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:00:07.976308 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:00:07.976308 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:00:07.976308 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:00:07.976308 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 321/632] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 12:00:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 364 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:07.981029 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0cf03b-04b1-4fed-a397-86a7bda18b92 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:08.005520 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd517bae-dc8c-4c96-ba5a-ba1064f2cf73 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:08.058564 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d563a5a0-9e82-4ba0-8a13-88c24e54ad6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:08.059696 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d563a5a0-9e82-4ba0-8a13-88c24e54ad6e None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:08.060049 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d563a5a0-9e82-4ba0-8a13-88c24e54ad6e None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:08.060650 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d563a5a0-9e82-4ba0-8a13-88c24e54ad6e None admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:08.061360 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d563a5a0-9e82-4ba0-8a13-88c24e54ad6e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:08.061731 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d563a5a0-9e82-4ba0-8a13-88c24e54ad6e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:08.064162 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d563a5a0-9e82-4ba0-8a13-88c24e54ad6e None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:08.082990 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 312/633] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 9 12:00:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:00:08.084148 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d563a5a0-9e82-4ba0-8a13-88c24e54ad6e None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:08.110370 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0cf03b-04b1-4fed-a397-86a7bda18b92 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:08.137395 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:08.148047 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938e94a6-c141-4f96-9e9c-e958c2fea7fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:08.148725 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938e94a6-c141-4f96-9e9c-e958c2fea7fc None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:08.148936 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938e94a6-c141-4f96-9e9c-e958c2fea7fc None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:08.149199 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938e94a6-c141-4f96-9e9c-e958c2fea7fc None admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:08.149900 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938e94a6-c141-4f96-9e9c-e958c2fea7fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:08.150146 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938e94a6-c141-4f96-9e9c-e958c2fea7fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:08.151559 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938e94a6-c141-4f96-9e9c-e958c2fea7fc None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:08.163735 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 322/634] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 9 12:00:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:08.168730 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938e94a6-c141-4f96-9e9c-e958c2fea7fc None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:08.199662 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:08.203805 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d563a5a0-9e82-4ba0-8a13-88c24e54ad6e None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:08.241485 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976ed5ff-8398-4581-a630-90d131772114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:08.243058 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976ed5ff-8398-4581-a630-90d131772114 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:08.243561 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976ed5ff-8398-4581-a630-90d131772114 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:08.244054 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976ed5ff-8398-4581-a630-90d131772114 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:08.257741 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976ed5ff-8398-4581-a630-90d131772114 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:00:08.264400 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-976ed5ff-8398-4581-a630-90d131772114 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 12:00:08.264400 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:08.264400 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:08.264400 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:08.264400 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:08.264400 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:08.264400 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:08.264400 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:08.264400 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:08.264400 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:08.264400 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:08.264400 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:08.264400 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:00:08.264400 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:00:08.264400 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:00:08.264400 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:00:08.265334 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:00:08.265334 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:00:08.265334 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:00:08.265334 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:00:08.265334 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:08.265334 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:08.265334 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:08.265334 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:08.265334 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:08.265334 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:08.265334 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:08.265334 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:08.265334 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:08.265334 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:08.265334 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:08.265334 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:08.266161 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:08.266161 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:08.266161 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:08.266161 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:08.266161 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:00:08.266161 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:00:08.266161 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:00:08.266161 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:00:08.266161 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:00:08.266161 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:00:08.266161 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 12:00:08.266161 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:00:08.273835 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 313/635] 10.222.0.49 () {56 vars in 1102 bytes} [Fri Sep 9 12:00:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:08.273835 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38840e53-d540-4eb3-912c-575fe92bd592 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:08.273835 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38840e53-d540-4eb3-912c-575fe92bd592 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:08.273835 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38840e53-d540-4eb3-912c-575fe92bd592 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:08.273835 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 314/636] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:00:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:00:08.273835 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38840e53-d540-4eb3-912c-575fe92bd592 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:08.288088 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938e94a6-c141-4f96-9e9c-e958c2fea7fc None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:08.324758 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec0230b-4356-40dd-9eb4-b1cbc13d68a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:08.325389 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec0230b-4356-40dd-9eb4-b1cbc13d68a0 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:08.325672 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec0230b-4356-40dd-9eb4-b1cbc13d68a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:08.325946 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec0230b-4356-40dd-9eb4-b1cbc13d68a0 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:08.333466 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec0230b-4356-40dd-9eb4-b1cbc13d68a0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:08.334958 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec0230b-4356-40dd-9eb4-b1cbc13d68a0 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:08.348647 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-3ec0230b-4356-40dd-9eb4-b1cbc13d68a0 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 09 12:00:08.348647 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:08.348647 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:08.348647 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:08.348647 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:08.348647 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:08.348647 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:08.348647 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:08.348647 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:08.348647 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:08.348647 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:08.348647 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:08.348647 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 12:00:08.348647 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 12:00:08.348647 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 12:00:08.348647 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 12:00:08.349700 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:08.349700 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:08.349700 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 12:00:08.349700 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:00:08.349700 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 12:00:08.349700 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:00:08.349700 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:08.349700 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:08.349700 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:08.349700 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:08.349700 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:08.349700 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:08.349700 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:08.349700 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:08.349700 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:08.349700 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:08.349700 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:08.349700 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:08.350662 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:08.350662 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:08.350662 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 12:00:08.350662 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 12:00:08.350662 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 12:00:08.350662 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 12:00:08.350662 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 12:00:08.350662 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 12:00:08.350662 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 09 12:00:08.350662 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:00:08.350662 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 323/637] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 9 12:00:08 2022] GET /identity/v3/users/glance => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:08.350662 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec0230b-4356-40dd-9eb4-b1cbc13d68a0 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:08.389979 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38840e53-d540-4eb3-912c-575fe92bd592 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:08.399897 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:08.429573 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655d7421-c38d-46d4-b551-59ad9cf438c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:08.430136 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655d7421-c38d-46d4-b551-59ad9cf438c2 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:08.430417 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655d7421-c38d-46d4-b551-59ad9cf438c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:08.430687 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655d7421-c38d-46d4-b551-59ad9cf438c2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:08.438946 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655d7421-c38d-46d4-b551-59ad9cf438c2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:08.440659 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655d7421-c38d-46d4-b551-59ad9cf438c2 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:08.453942 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-655d7421-c38d-46d4-b551-59ad9cf438c2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 12:00:08.453942 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:08.453942 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:08.453942 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:08.453942 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:08.453942 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:08.453942 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:08.453942 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:08.453942 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:08.453942 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:08.453942 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:08.453942 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:08.453942 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 12:00:08.453942 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 12:00:08.453942 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 12:00:08.453942 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 12:00:08.454981 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:08.454981 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:08.454981 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 12:00:08.454981 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:00:08.454981 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 12:00:08.454981 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:00:08.454981 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:08.454981 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:08.454981 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:08.454981 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:08.454981 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:08.454981 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:08.454981 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:08.454981 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:08.454981 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:08.454981 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:08.454981 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:08.454981 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:08.457034 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:08.457034 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:08.457034 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 12:00:08.457034 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 12:00:08.457034 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 12:00:08.457034 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 12:00:08.457034 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 12:00:08.457034 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 12:00:08.457034 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 12:00:08.457034 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:00:08.457034 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 315/638] 10.222.0.49 () {56 vars in 1093 bytes} [Fri Sep 9 12:00:08 2022] GET /identity/v3/users/nova => generated 82 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:08.460140 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec0230b-4356-40dd-9eb4-b1cbc13d68a0 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:08.460751 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da68f736-9e0e-49c3-afa8-b81c565a6f2f None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:08.461233 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da68f736-9e0e-49c3-afa8-b81c565a6f2f None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:08.461839 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da68f736-9e0e-49c3-afa8-b81c565a6f2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:08.480881 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-da68f736-9e0e-49c3-afa8-b81c565a6f2f None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:00:08.482087 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-da68f736-9e0e-49c3-afa8-b81c565a6f2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:08.496472 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:08.498190 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4abafd-c041-4d1a-9f50-8eb2cde87ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:08.498616 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4abafd-c041-4d1a-9f50-8eb2cde87ee9 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:08.498794 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4abafd-c041-4d1a-9f50-8eb2cde87ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:08.499174 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4abafd-c041-4d1a-9f50-8eb2cde87ee9 None admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:08.500016 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4abafd-c041-4d1a-9f50-8eb2cde87ee9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:08.500321 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4abafd-c041-4d1a-9f50-8eb2cde87ee9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:08.501714 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4abafd-c041-4d1a-9f50-8eb2cde87ee9 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:08.515202 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 324/639] 10.222.0.49 () {56 vars in 1103 bytes} [Fri Sep 9 12:00:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:08.515202 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4abafd-c041-4d1a-9f50-8eb2cde87ee9 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:08.557914 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 316/640] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:00:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 12:00:08.563090 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da68f736-9e0e-49c3-afa8-b81c565a6f2f None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:08.642030 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4abafd-c041-4d1a-9f50-8eb2cde87ee9 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:08.656371 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:08.679831 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da68f736-9e0e-49c3-afa8-b81c565a6f2f None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:08.690120 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09560cf-8b03-4fbb-928b-eee906e0cca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:08.691068 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09560cf-8b03-4fbb-928b-eee906e0cca4 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:08.691707 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09560cf-8b03-4fbb-928b-eee906e0cca4 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:08.692202 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09560cf-8b03-4fbb-928b-eee906e0cca4 None admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:08.693166 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09560cf-8b03-4fbb-928b-eee906e0cca4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:08.693678 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09560cf-8b03-4fbb-928b-eee906e0cca4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:08.696390 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09560cf-8b03-4fbb-928b-eee906e0cca4 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:08.716669 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c8293b-7366-4ff2-81af-9560849842ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:08.717275 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c8293b-7366-4ff2-81af-9560849842ec None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:08.717551 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c8293b-7366-4ff2-81af-9560849842ec None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:08.717820 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c8293b-7366-4ff2-81af-9560849842ec None admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:08.718665 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c8293b-7366-4ff2-81af-9560849842ec None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:08.719077 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c8293b-7366-4ff2-81af-9560849842ec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:08.719579 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 325/641] 10.222.0.49 () {56 vars in 1137 bytes} [Fri Sep 9 12:00:08 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:08.720761 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09560cf-8b03-4fbb-928b-eee906e0cca4 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:08.725596 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c8293b-7366-4ff2-81af-9560849842ec None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:08.750108 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 317/642] 10.222.0.49 () {56 vars in 1133 bytes} [Fri Sep 9 12:00:08 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:08.751057 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c8293b-7366-4ff2-81af-9560849842ec None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:08.760528 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:08.845496 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09560cf-8b03-4fbb-928b-eee906e0cca4 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:08.860408 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c8293b-7366-4ff2-81af-9560849842ec None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:08.888415 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e20f6c0-6640-4000-b756-498dc2c5b50f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:08.889857 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e20f6c0-6640-4000-b756-498dc2c5b50f None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:08.890442 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e20f6c0-6640-4000-b756-498dc2c5b50f None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:08.890931 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e20f6c0-6640-4000-b756-498dc2c5b50f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:08.897279 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcbaa84-51a8-43f7-bbc5-9b0b9aef9b79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:08.897882 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcbaa84-51a8-43f7-bbc5-9b0b9aef9b79 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:08.898160 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcbaa84-51a8-43f7-bbc5-9b0b9aef9b79 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:08.898433 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcbaa84-51a8-43f7-bbc5-9b0b9aef9b79 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:08.903798 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e20f6c0-6640-4000-b756-498dc2c5b50f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:08.905434 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e20f6c0-6640-4000-b756-498dc2c5b50f None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:08.921787 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-8e20f6c0-6640-4000-b756-498dc2c5b50f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:00:08.921787 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:08.921787 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:08.921787 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:08.921787 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:08.921787 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:08.921787 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:08.921787 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:08.921787 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:08.921787 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:08.921787 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:08.921787 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:08.921787 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:00:08.921787 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:00:08.921787 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:00:08.921787 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:00:08.922816 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:08.922816 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:08.922816 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:08.922816 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:08.922816 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:08.922816 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:08.922816 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:08.922816 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:08.922816 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:08.922816 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:08.922816 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:08.922816 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:08.922816 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:08.922816 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:08.922816 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:08.922816 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:08.922816 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:00:08.923765 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:00:08.923765 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:00:08.923765 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:00:08.927488 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 326/643] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 12:00:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:08.928237 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e20f6c0-6640-4000-b756-498dc2c5b50f None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:08.948957 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:08.970332 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcbaa84-51a8-43f7-bbc5-9b0b9aef9b79 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:08.971926 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcbaa84-51a8-43f7-bbc5-9b0b9aef9b79 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:08.980600 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:09.037344 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e20f6c0-6640-4000-b756-498dc2c5b50f None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:09.076379 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f816c243-9117-44c9-aadb-d00d1a603a5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:09.076989 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f816c243-9117-44c9-aadb-d00d1a603a5c None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:09.077271 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f816c243-9117-44c9-aadb-d00d1a603a5c None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:09.077543 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f816c243-9117-44c9-aadb-d00d1a603a5c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:09.083462 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 318/644] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 9 12:00:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 12:00:09.083462 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcbaa84-51a8-43f7-bbc5-9b0b9aef9b79 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:09.150452 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f816c243-9117-44c9-aadb-d00d1a603a5c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:09.152439 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f816c243-9117-44c9-aadb-d00d1a603a5c None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:09.173692 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:09.192725 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcbaa84-51a8-43f7-bbc5-9b0b9aef9b79 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:09.200976 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:09.230504 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a53945-f723-446b-aefb-a8a3a6f6e2b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:09.231251 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a53945-f723-446b-aefb-a8a3a6f6e2b6 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:09.231644 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a53945-f723-446b-aefb-a8a3a6f6e2b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:09.231948 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a53945-f723-446b-aefb-a8a3a6f6e2b6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:09.242831 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a53945-f723-446b-aefb-a8a3a6f6e2b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:00:09.247736 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-a3a53945-f723-446b-aefb-a8a3a6f6e2b6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 12:00:09.247736 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:09.247736 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:09.247736 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:09.247736 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:09.247736 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:09.247736 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:09.247736 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:09.247736 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:09.247736 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:09.247736 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:09.247736 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:09.247736 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:00:09.247736 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:00:09.247736 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:00:09.247736 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:00:09.248675 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:00:09.248675 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:00:09.248675 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:00:09.248675 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:00:09.248675 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:09.248675 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:09.248675 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:09.248675 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:09.248675 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:09.248675 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:09.248675 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:09.248675 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:09.248675 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:09.248675 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:09.248675 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:09.248675 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:09.249650 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:09.249650 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:09.249650 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:09.249650 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:09.249650 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:00:09.249650 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:00:09.249650 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:00:09.249650 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:00:09.249650 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:00:09.249650 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:00:09.249650 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 12:00:09.249650 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:00:09.249650 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 319/645] 10.222.0.49 () {56 vars in 1102 bytes} [Fri Sep 9 12:00:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:09.249650 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a53945-f723-446b-aefb-a8a3a6f6e2b6 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:09.279114 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 327/646] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 9 12:00:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 12:00:09.279114 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f816c243-9117-44c9-aadb-d00d1a603a5c None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:09.361833 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a53945-f723-446b-aefb-a8a3a6f6e2b6 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:09.381120 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f816c243-9117-44c9-aadb-d00d1a603a5c None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:09.397872 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:09.401284 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cdafda-ef3c-457b-a02e-fdfaa3fc9dc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:09.402232 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cdafda-ef3c-457b-a02e-fdfaa3fc9dc4 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:09.403185 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cdafda-ef3c-457b-a02e-fdfaa3fc9dc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:09.403767 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cdafda-ef3c-457b-a02e-fdfaa3fc9dc4 None admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:09.404832 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cdafda-ef3c-457b-a02e-fdfaa3fc9dc4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:09.405331 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cdafda-ef3c-457b-a02e-fdfaa3fc9dc4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:09.406882 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cdafda-ef3c-457b-a02e-fdfaa3fc9dc4 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:09.414979 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5448b57-cb92-4e86-8dbe-7b1ceb0abe5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:09.415606 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5448b57-cb92-4e86-8dbe-7b1ceb0abe5e None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:09.415885 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5448b57-cb92-4e86-8dbe-7b1ceb0abe5e None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:09.416153 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5448b57-cb92-4e86-8dbe-7b1ceb0abe5e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:09.421738 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 320/647] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 9 12:00:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:09.422743 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cdafda-ef3c-457b-a02e-fdfaa3fc9dc4 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:09.430746 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5448b57-cb92-4e86-8dbe-7b1ceb0abe5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:09.432231 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5448b57-cb92-4e86-8dbe-7b1ceb0abe5e None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:09.448187 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-d5448b57-cb92-4e86-8dbe-7b1ceb0abe5e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:00:09.448187 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:09.448187 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:09.448187 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:09.448187 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:09.448187 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:09.448187 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:09.448187 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:09.448187 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:09.448187 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:09.448187 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:09.448187 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:09.448187 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:00:09.448187 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:00:09.448187 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:00:09.448187 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:00:09.449357 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:09.449357 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:09.449357 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:09.449357 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:09.449357 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:09.449357 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:09.449357 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:09.449357 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:09.449357 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:09.449357 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:09.449357 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:09.449357 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:09.449357 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:09.449357 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:09.449357 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:09.449357 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:09.449357 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:00:09.450451 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:00:09.450451 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:00:09.450451 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:00:09.453899 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 328/648] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 12:00:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:09.455048 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5448b57-cb92-4e86-8dbe-7b1ceb0abe5e None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:09.458556 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:09.542042 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cdafda-ef3c-457b-a02e-fdfaa3fc9dc4 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:09.567014 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5448b57-cb92-4e86-8dbe-7b1ceb0abe5e None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:09.576981 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0106f110-31b4-4dfd-9878-9e8a1d910e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:09.577589 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0106f110-31b4-4dfd-9878-9e8a1d910e19 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:09.577868 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0106f110-31b4-4dfd-9878-9e8a1d910e19 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:09.578139 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0106f110-31b4-4dfd-9878-9e8a1d910e19 None admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:09.578798 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0106f110-31b4-4dfd-9878-9e8a1d910e19 None admin] RBAC: Adding query filter params (name=service) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:09.579143 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0106f110-31b4-4dfd-9878-9e8a1d910e19 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:09.580507 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0106f110-31b4-4dfd-9878-9e8a1d910e19 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:09.589215 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 321/649] 10.222.0.49 () {56 vars in 1103 bytes} [Fri Sep 9 12:00:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:09.589818 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0106f110-31b4-4dfd-9878-9e8a1d910e19 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:09.606566 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b9777d-782d-4925-afd5-1dc67ac5b6a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:09.607235 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b9777d-782d-4925-afd5-1dc67ac5b6a0 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:09.607513 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b9777d-782d-4925-afd5-1dc67ac5b6a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:09.607779 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b9777d-782d-4925-afd5-1dc67ac5b6a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:09.619146 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b9777d-782d-4925-afd5-1dc67ac5b6a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:09.620590 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b9777d-782d-4925-afd5-1dc67ac5b6a0 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:09.635660 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-f7b9777d-782d-4925-afd5-1dc67ac5b6a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:00:09.635660 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:09.635660 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:09.635660 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:09.635660 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:09.635660 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:09.635660 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:09.635660 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:09.635660 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:09.635660 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:09.635660 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:09.635660 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:09.635660 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:00:09.635660 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:00:09.635660 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:00:09.635660 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:00:09.636644 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:09.636644 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:09.636644 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:09.636644 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:09.636644 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:09.636644 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:09.636644 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:09.636644 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:09.636644 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:09.636644 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:09.636644 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:09.636644 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:09.636644 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:09.636644 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:09.636644 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:09.636644 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:09.636644 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:00:09.637522 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:00:09.637522 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:00:09.637522 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:00:09.637522 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 329/650] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 12:00:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:09.637522 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b9777d-782d-4925-afd5-1dc67ac5b6a0 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:09.662419 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:09.684818 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:09.698259 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0106f110-31b4-4dfd-9878-9e8a1d910e19 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:09.731617 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f91d805-69c8-4630-9ddc-0a06207c82d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:09.732173 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f91d805-69c8-4630-9ddc-0a06207c82d7 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:09.732444 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f91d805-69c8-4630-9ddc-0a06207c82d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:09.732706 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f91d805-69c8-4630-9ddc-0a06207c82d7 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:09.738080 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b9777d-782d-4925-afd5-1dc67ac5b6a0 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:09.740350 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f91d805-69c8-4630-9ddc-0a06207c82d7 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:09.741803 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f91d805-69c8-4630-9ddc-0a06207c82d7 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:09.752774 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-7f91d805-69c8-4630-9ddc-0a06207c82d7 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 12:00:09.752774 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:09.752774 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:09.752774 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:09.752774 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:09.752774 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:09.752774 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:09.752774 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:09.752774 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:09.752774 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:09.752774 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:09.752774 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:09.752774 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 12:00:09.752774 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 12:00:09.752774 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 12:00:09.752774 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 12:00:09.753823 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:09.753823 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:09.753823 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 12:00:09.753823 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:00:09.753823 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 12:00:09.753823 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:00:09.753823 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:09.753823 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:09.753823 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:09.753823 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:09.753823 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:09.753823 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:09.753823 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:09.753823 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:09.753823 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:09.753823 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:09.753823 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:09.753823 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:09.754854 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:09.754854 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:09.754854 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 12:00:09.754854 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 12:00:09.754854 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 12:00:09.754854 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 12:00:09.754854 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 12:00:09.754854 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 12:00:09.754854 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 12:00:09.754854 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:00:09.754854 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 322/651] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 9 12:00:09 2022] GET /identity/v3/users/cinder => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:09.754854 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f91d805-69c8-4630-9ddc-0a06207c82d7 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:09.777044 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c2f4bb-b5cd-43d0-88e0-e782dce2f09f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:09.777652 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c2f4bb-b5cd-43d0-88e0-e782dce2f09f None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:09.777930 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c2f4bb-b5cd-43d0-88e0-e782dce2f09f None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:09.778200 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c2f4bb-b5cd-43d0-88e0-e782dce2f09f None admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:09.778836 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c2f4bb-b5cd-43d0-88e0-e782dce2f09f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:09.779165 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c2f4bb-b5cd-43d0-88e0-e782dce2f09f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:09.780522 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c2f4bb-b5cd-43d0-88e0-e782dce2f09f None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:09.793041 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 330/652] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 9 12:00:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:09.794221 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb95715-ffc5-40d7-990d-f9e8e5e8efd6 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:09.794506 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb95715-ffc5-40d7-990d-f9e8e5e8efd6 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:09.794774 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb95715-ffc5-40d7-990d-f9e8e5e8efd6 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:09.796003 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 331/653] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:00:09 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:00:09.796691 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb95715-ffc5-40d7-990d-f9e8e5e8efd6 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:09.858912 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f91d805-69c8-4630-9ddc-0a06207c82d7 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:09.881842 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:09.892574 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802ca5b1-5fb5-47d9-b83d-b411dffc10b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:09.893203 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802ca5b1-5fb5-47d9-b83d-b411dffc10b3 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:09.893479 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802ca5b1-5fb5-47d9-b83d-b411dffc10b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:09.893750 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802ca5b1-5fb5-47d9-b83d-b411dffc10b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:09.901604 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb95715-ffc5-40d7-990d-f9e8e5e8efd6 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:09.904687 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802ca5b1-5fb5-47d9-b83d-b411dffc10b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:09.906123 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802ca5b1-5fb5-47d9-b83d-b411dffc10b3 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:09.914607 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:09.921276 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-802ca5b1-5fb5-47d9-b83d-b411dffc10b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:00:09.921276 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:09.921276 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:09.921276 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:09.921276 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:09.921276 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:09.921276 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:09.921276 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:09.921276 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:09.921276 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:09.921276 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:09.921276 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:09.921276 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:00:09.921276 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:00:09.921276 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:00:09.921276 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:00:09.922426 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:09.922426 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:09.922426 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:09.922426 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:09.922426 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:09.922426 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:09.922426 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:09.922426 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:09.922426 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:09.922426 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:09.922426 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:09.922426 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:09.922426 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:09.922426 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:09.922426 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:09.922426 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:09.922426 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:00:09.923685 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:00:09.923685 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:00:09.923685 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:00:09.923685 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 323/654] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 12:00:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:09.923685 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802ca5b1-5fb5-47d9-b83d-b411dffc10b3 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:09.937957 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97addcf8-3655-452b-af78-588385132eb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:09.938562 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97addcf8-3655-452b-af78-588385132eb7 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:09.938842 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97addcf8-3655-452b-af78-588385132eb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:09.939146 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97addcf8-3655-452b-af78-588385132eb7 None admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:09.940043 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97addcf8-3655-452b-af78-588385132eb7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:09.940344 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97addcf8-3655-452b-af78-588385132eb7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:09.942236 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97addcf8-3655-452b-af78-588385132eb7 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:09.953666 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 332/655] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 9 12:00:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:09.954446 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97addcf8-3655-452b-af78-588385132eb7 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:10.027263 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802ca5b1-5fb5-47d9-b83d-b411dffc10b3 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:10.052065 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97addcf8-3655-452b-af78-588385132eb7 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:10.060091 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4444951-74b9-4bbf-a72e-eb5be8ee236c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:10.060684 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4444951-74b9-4bbf-a72e-eb5be8ee236c None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:10.060959 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4444951-74b9-4bbf-a72e-eb5be8ee236c None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:10.061226 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4444951-74b9-4bbf-a72e-eb5be8ee236c None admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:10.061959 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4444951-74b9-4bbf-a72e-eb5be8ee236c None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:10.062260 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4444951-74b9-4bbf-a72e-eb5be8ee236c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:10.063715 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4444951-74b9-4bbf-a72e-eb5be8ee236c None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:10.081414 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 324/656] 10.222.0.49 () {56 vars in 1137 bytes} [Fri Sep 9 12:00:09 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:10.083534 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1204b3e6-ce6f-4ac0-a0ea-5075888ec434 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:10.083817 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1204b3e6-ce6f-4ac0-a0ea-5075888ec434 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:10.084098 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1204b3e6-ce6f-4ac0-a0ea-5075888ec434 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:10.087273 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813d9d17-b87f-4140-8960-0d1e64434892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:10.087900 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813d9d17-b87f-4140-8960-0d1e64434892 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:10.088173 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813d9d17-b87f-4140-8960-0d1e64434892 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:10.088439 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813d9d17-b87f-4140-8960-0d1e64434892 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:10.095655 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813d9d17-b87f-4140-8960-0d1e64434892 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:10.097073 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813d9d17-b87f-4140-8960-0d1e64434892 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:10.101922 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-1204b3e6-ce6f-4ac0-a0ea-5075888ec434 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:00:10.103151 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-1204b3e6-ce6f-4ac0-a0ea-5075888ec434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:10.108165 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-813d9d17-b87f-4140-8960-0d1e64434892 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:00:10.108165 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:10.108165 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:10.108165 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:10.108165 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:10.108165 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:10.108165 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:10.108165 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:10.108165 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:10.108165 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:10.108165 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:10.108165 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:10.108165 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 12:00:10.108165 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 12:00:10.108165 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 12:00:10.109166 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 12:00:10.109166 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:10.109166 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:10.109166 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:10.109166 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:10.109166 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:10.109166 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:10.109166 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:10.109166 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:10.109166 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:10.109166 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:10.109166 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:10.109166 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:10.109166 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:10.109166 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:10.109166 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:10.109166 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:10.110070 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 12:00:10.110070 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 12:00:10.110070 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 12:00:10.110070 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 12:00:10.110070 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 12:00:10.110070 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 12:00:10.110070 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:00:10.110070 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:00:10.110070 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 333/657] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 9 12:00:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:10.110070 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813d9d17-b87f-4140-8960-0d1e64434892 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:10.116325 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:10.159465 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:10.173643 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 325/658] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:00:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 12:00:10.174466 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1204b3e6-ce6f-4ac0-a0ea-5075888ec434 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:10.228396 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813d9d17-b87f-4140-8960-0d1e64434892 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:10.262921 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a149579f-ec4a-468e-9df1-81d26f61fa2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:10.263677 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a149579f-ec4a-468e-9df1-81d26f61fa2a None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:10.263962 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a149579f-ec4a-468e-9df1-81d26f61fa2a None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:10.264229 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a149579f-ec4a-468e-9df1-81d26f61fa2a None admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:10.264857 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a149579f-ec4a-468e-9df1-81d26f61fa2a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:10.265153 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a149579f-ec4a-468e-9df1-81d26f61fa2a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:10.266505 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a149579f-ec4a-468e-9df1-81d26f61fa2a None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:10.283918 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 334/659] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 9 12:00:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:10.283918 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a149579f-ec4a-468e-9df1-81d26f61fa2a None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:10.288240 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1204b3e6-ce6f-4ac0-a0ea-5075888ec434 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:10.322671 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3330114b-725a-4a5d-ac05-34e9e61f0674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:10.323355 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3330114b-725a-4a5d-ac05-34e9e61f0674 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:10.323637 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3330114b-725a-4a5d-ac05-34e9e61f0674 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:10.323911 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3330114b-725a-4a5d-ac05-34e9e61f0674 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:10.330538 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3330114b-725a-4a5d-ac05-34e9e61f0674 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:10.332576 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3330114b-725a-4a5d-ac05-34e9e61f0674 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:10.344694 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-3330114b-725a-4a5d-ac05-34e9e61f0674 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:00:10.344694 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:10.344694 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:10.344694 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:10.344694 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:10.344694 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:10.344694 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:10.344694 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:10.344694 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:10.344694 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:10.344694 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:10.344694 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:10.344694 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 12:00:10.344694 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 12:00:10.344694 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 12:00:10.345583 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 12:00:10.345583 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:10.345583 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:10.345583 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:10.345583 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:10.345583 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:10.345583 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:10.345583 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:10.345583 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:10.345583 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:10.345583 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:10.345583 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:10.345583 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:10.345583 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:10.345583 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:10.345583 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:10.345583 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:10.346485 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 12:00:10.346485 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 12:00:10.346485 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 12:00:10.346485 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 12:00:10.346485 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 12:00:10.346485 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 12:00:10.346485 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:00:10.346485 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:00:10.346485 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 326/660] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 9 12:00:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:10.347243 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3330114b-725a-4a5d-ac05-34e9e61f0674 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:10.348401 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:10.391272 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a149579f-ec4a-468e-9df1-81d26f61fa2a None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:10.405233 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:10.429256 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f48008-d577-4b77-a0ca-bf8578e125f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:10.430023 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f48008-d577-4b77-a0ca-bf8578e125f4 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:10.431112 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f48008-d577-4b77-a0ca-bf8578e125f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:10.431394 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f48008-d577-4b77-a0ca-bf8578e125f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:10.460004 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3330114b-725a-4a5d-ac05-34e9e61f0674 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:10.496879 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12819c91-bdda-46ed-9f77-13f5873a5b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:10.499640 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12819c91-bdda-46ed-9f77-13f5873a5b49 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:10.499924 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12819c91-bdda-46ed-9f77-13f5873a5b49 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:10.500199 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12819c91-bdda-46ed-9f77-13f5873a5b49 None admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:10.500892 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12819c91-bdda-46ed-9f77-13f5873a5b49 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:10.501709 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12819c91-bdda-46ed-9f77-13f5873a5b49 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:10.503207 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12819c91-bdda-46ed-9f77-13f5873a5b49 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:10.507516 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f48008-d577-4b77-a0ca-bf8578e125f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:10.510053 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f48008-d577-4b77-a0ca-bf8578e125f4 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:10.517424 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 327/661] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Sep 9 12:00:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:10.518258 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12819c91-bdda-46ed-9f77-13f5873a5b49 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:10.586623 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:10.626224 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 335/662] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 9 12:00:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 12:00:10.627101 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f48008-d577-4b77-a0ca-bf8578e125f4 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:10.635806 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12819c91-bdda-46ed-9f77-13f5873a5b49 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:10.646445 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:10.673602 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27cdcae-db01-492e-ad63-d4b89a0a5f54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:10.674230 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27cdcae-db01-492e-ad63-d4b89a0a5f54 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:10.674503 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27cdcae-db01-492e-ad63-d4b89a0a5f54 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:10.674769 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27cdcae-db01-492e-ad63-d4b89a0a5f54 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:10.683658 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27cdcae-db01-492e-ad63-d4b89a0a5f54 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:10.685102 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27cdcae-db01-492e-ad63-d4b89a0a5f54 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:10.696968 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-c27cdcae-db01-492e-ad63-d4b89a0a5f54 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 09 12:00:10.696968 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:10.696968 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:10.696968 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:10.696968 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:10.696968 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:10.696968 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:10.696968 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:10.696968 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:10.696968 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:10.696968 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:10.696968 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:10.696968 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 12:00:10.696968 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 12:00:10.696968 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 12:00:10.696968 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 12:00:10.698099 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:10.698099 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:10.698099 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 12:00:10.698099 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:00:10.698099 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 12:00:10.698099 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:00:10.698099 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:10.698099 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:10.698099 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:10.698099 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:10.698099 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:10.698099 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:10.698099 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:10.698099 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:10.698099 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:10.698099 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:10.698099 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:10.698099 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:10.699282 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:10.699282 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:10.699282 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 12:00:10.699282 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 12:00:10.699282 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 12:00:10.699282 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 12:00:10.699282 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 12:00:10.699282 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 12:00:10.699282 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 09 12:00:10.699282 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:00:10.699282 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 328/663] 10.222.0.49 () {56 vars in 1102 bytes} [Fri Sep 9 12:00:10 2022] GET /identity/v3/users/neutron => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:10.699282 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27cdcae-db01-492e-ad63-d4b89a0a5f54 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:10.734753 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f48008-d577-4b77-a0ca-bf8578e125f4 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:10.768988 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0cb30d-c405-4f64-90e2-d86a5f3f4e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:10.770419 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0cb30d-c405-4f64-90e2-d86a5f3f4e25 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:10.770885 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0cb30d-c405-4f64-90e2-d86a5f3f4e25 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:10.771224 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0cb30d-c405-4f64-90e2-d86a5f3f4e25 None admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:10.771906 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0cb30d-c405-4f64-90e2-d86a5f3f4e25 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:10.772206 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0cb30d-c405-4f64-90e2-d86a5f3f4e25 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:10.773625 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0cb30d-c405-4f64-90e2-d86a5f3f4e25 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:10.784569 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 336/664] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Sep 9 12:00:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:10.785645 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0cb30d-c405-4f64-90e2-d86a5f3f4e25 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:10.801532 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27cdcae-db01-492e-ad63-d4b89a0a5f54 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:10.812187 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:10.836035 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013780ca-c81b-4f57-a334-a547e0adcfe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:10.836710 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013780ca-c81b-4f57-a334-a547e0adcfe2 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:10.836939 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013780ca-c81b-4f57-a334-a547e0adcfe2 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:10.837207 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013780ca-c81b-4f57-a334-a547e0adcfe2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:10.846959 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013780ca-c81b-4f57-a334-a547e0adcfe2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:00:10.852350 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-013780ca-c81b-4f57-a334-a547e0adcfe2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 12:00:10.852350 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:10.852350 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:10.852350 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:10.852350 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:10.852350 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:10.852350 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:10.852350 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:10.852350 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:10.852350 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:10.852350 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:10.852350 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:10.852350 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:00:10.852350 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:00:10.852350 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:00:10.852350 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:00:10.853486 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:00:10.853486 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:00:10.853486 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:00:10.853486 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:00:10.853486 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:10.853486 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:10.853486 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:10.853486 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:10.853486 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:10.853486 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:10.853486 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:10.853486 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:10.853486 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:10.853486 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:10.853486 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:10.853486 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:10.854328 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:10.854328 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:10.854328 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:10.854328 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:10.854328 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:00:10.854328 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:00:10.854328 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:00:10.854328 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:00:10.854328 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:00:10.854328 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:00:10.854328 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 12:00:10.854328 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:00:10.854328 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 329/665] 10.222.0.49 () {56 vars in 1102 bytes} [Fri Sep 9 12:00:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:10.855054 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013780ca-c81b-4f57-a334-a547e0adcfe2 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:10.883183 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:10.892323 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0cb30d-c405-4f64-90e2-d86a5f3f4e25 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:10.926052 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e946245-523d-422d-9cd1-961819a9e519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:10.926744 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e946245-523d-422d-9cd1-961819a9e519 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:10.927056 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e946245-523d-422d-9cd1-961819a9e519 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:10.927326 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e946245-523d-422d-9cd1-961819a9e519 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:10.928041 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e946245-523d-422d-9cd1-961819a9e519 None admin] RBAC: Adding query filter params (role.id=8abeff2cb87d4b7fa72b27b028005877, scope.project.id=fe863fcd75f1406ebbffc5016078f73e, user.id=37c4e14aa8e04d3e85a406b9363bca2a) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:10.928343 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e946245-523d-422d-9cd1-961819a9e519 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:10.929761 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e946245-523d-422d-9cd1-961819a9e519 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:10.935529 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 337/666] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Sep 9 12:00:10 2022] GET /identity/v3/role_assignments?user.id=37c4e14aa8e04d3e85a406b9363bca2a&scope.project.id=fe863fcd75f1406ebbffc5016078f73e&role.id=8abeff2cb87d4b7fa72b27b028005877 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:10.936309 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e946245-523d-422d-9cd1-961819a9e519 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:10.974515 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013780ca-c81b-4f57-a334-a547e0adcfe2 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:11.012760 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df53ed8-be47-4aeb-8cd8-48c41072376b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:11.013502 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df53ed8-be47-4aeb-8cd8-48c41072376b None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:11.013695 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df53ed8-be47-4aeb-8cd8-48c41072376b None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:11.013947 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df53ed8-be47-4aeb-8cd8-48c41072376b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:11.028572 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df53ed8-be47-4aeb-8cd8-48c41072376b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:11.030090 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df53ed8-be47-4aeb-8cd8-48c41072376b None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:11.044111 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:11.047229 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-7df53ed8-be47-4aeb-8cd8-48c41072376b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:00:11.047229 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:11.047229 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:11.047229 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:11.047229 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:11.047229 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:11.047229 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:11.047229 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:11.047229 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:11.047229 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:11.047229 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:11.047229 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:11.047229 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:00:11.047229 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:00:11.047229 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:00:11.047229 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:00:11.048380 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:11.048380 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:11.048380 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:11.048380 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:11.048380 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:11.048380 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:11.048380 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:11.048380 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:11.048380 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:11.048380 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:11.048380 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:11.048380 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:11.048380 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:11.048380 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:11.048380 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:11.048380 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:11.048380 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:00:11.049315 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:00:11.049315 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:00:11.049315 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:00:11.049315 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 330/667] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 12:00:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:11.049315 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df53ed8-be47-4aeb-8cd8-48c41072376b None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:11.067972 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e946245-523d-422d-9cd1-961819a9e519 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:11.106207 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780f376a-7834-478e-8a45-407d3558de88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:11.106731 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780f376a-7834-478e-8a45-407d3558de88 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:11.107039 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780f376a-7834-478e-8a45-407d3558de88 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:11.107334 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780f376a-7834-478e-8a45-407d3558de88 None admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:11.108268 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780f376a-7834-478e-8a45-407d3558de88 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:11.108461 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780f376a-7834-478e-8a45-407d3558de88 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:11.109885 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780f376a-7834-478e-8a45-407d3558de88 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:11.131547 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 338/668] 10.222.0.49 () {56 vars in 1139 bytes} [Fri Sep 9 12:00:10 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:11.132416 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780f376a-7834-478e-8a45-407d3558de88 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:11.145829 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:11.166216 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df53ed8-be47-4aeb-8cd8-48c41072376b None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:11.209199 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fef9c0e-1403-4807-aef6-be2f9ea30c19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:11.209707 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fef9c0e-1403-4807-aef6-be2f9ea30c19 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:11.209985 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fef9c0e-1403-4807-aef6-be2f9ea30c19 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:11.210255 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fef9c0e-1403-4807-aef6-be2f9ea30c19 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:11.211410 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fef9c0e-1403-4807-aef6-be2f9ea30c19 None admin] RBAC: Adding query filter params (role.id=8abeff2cb87d4b7fa72b27b028005877, scope.project.id=fe863fcd75f1406ebbffc5016078f73e, user.id=70bac169d5cd43e1926f310344ac11fa) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:11.211410 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fef9c0e-1403-4807-aef6-be2f9ea30c19 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:11.212742 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fef9c0e-1403-4807-aef6-be2f9ea30c19 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:11.218534 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 331/669] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Sep 9 12:00:11 2022] GET /identity/v3/role_assignments?user.id=70bac169d5cd43e1926f310344ac11fa&scope.project.id=fe863fcd75f1406ebbffc5016078f73e&role.id=8abeff2cb87d4b7fa72b27b028005877 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:11.219450 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fef9c0e-1403-4807-aef6-be2f9ea30c19 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:11.255901 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780f376a-7834-478e-8a45-407d3558de88 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:11.298360 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385d71c9-0d4f-442f-af99-d3758b36bb88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:11.299016 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385d71c9-0d4f-442f-af99-d3758b36bb88 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:11.299694 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385d71c9-0d4f-442f-af99-d3758b36bb88 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:11.299990 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385d71c9-0d4f-442f-af99-d3758b36bb88 None admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:11.301292 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385d71c9-0d4f-442f-af99-d3758b36bb88 None admin] RBAC: Adding query filter params (name=service) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:11.301686 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385d71c9-0d4f-442f-af99-d3758b36bb88 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:11.303102 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385d71c9-0d4f-442f-af99-d3758b36bb88 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:11.318436 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 339/670] 10.222.0.49 () {56 vars in 1103 bytes} [Fri Sep 9 12:00:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:11.318436 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385d71c9-0d4f-442f-af99-d3758b36bb88 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:11.328703 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:11.393618 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fef9c0e-1403-4807-aef6-be2f9ea30c19 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:11.444406 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c6de17-e9d0-434a-8fa2-709e069e86b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:11.445231 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c6de17-e9d0-434a-8fa2-709e069e86b9 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:11.445360 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c6de17-e9d0-434a-8fa2-709e069e86b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:11.445568 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c6de17-e9d0-434a-8fa2-709e069e86b9 None admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:11.446210 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c6de17-e9d0-434a-8fa2-709e069e86b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:11.446515 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c6de17-e9d0-434a-8fa2-709e069e86b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:11.447935 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c6de17-e9d0-434a-8fa2-709e069e86b9 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:11.448787 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385d71c9-0d4f-442f-af99-d3758b36bb88 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:11.467048 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 332/671] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 9 12:00:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:11.467960 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c6de17-e9d0-434a-8fa2-709e069e86b9 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:11.474061 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:11.493854 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0b0e6f-c516-457e-88cb-6316ee6c2aa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:11.494473 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0b0e6f-c516-457e-88cb-6316ee6c2aa4 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:11.494751 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0b0e6f-c516-457e-88cb-6316ee6c2aa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:11.495077 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0b0e6f-c516-457e-88cb-6316ee6c2aa4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:11.577696 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0b0e6f-c516-457e-88cb-6316ee6c2aa4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:11.579321 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0b0e6f-c516-457e-88cb-6316ee6c2aa4 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:11.587740 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:11.597063 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c6de17-e9d0-434a-8fa2-709e069e86b9 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:11.649196 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d248f263-8229-45c1-aab4-02da0f4e0303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:11.650090 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d248f263-8229-45c1-aab4-02da0f4e0303 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:11.650467 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d248f263-8229-45c1-aab4-02da0f4e0303 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:11.650876 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d248f263-8229-45c1-aab4-02da0f4e0303 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:11.666067 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d248f263-8229-45c1-aab4-02da0f4e0303 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:11.668166 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d248f263-8229-45c1-aab4-02da0f4e0303 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:11.687906 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-d248f263-8229-45c1-aab4-02da0f4e0303 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:00:11.687906 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:11.687906 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:11.687906 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:11.687906 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:11.687906 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:11.687906 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:11.687906 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:11.687906 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:11.687906 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:11.687906 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:11.687906 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:11.687906 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:00:11.687906 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:00:11.687906 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:00:11.687906 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:00:11.689595 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:11.689595 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:11.689595 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:11.689595 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:11.689595 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:11.689595 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:11.689595 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:11.689595 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:11.689595 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:11.689595 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:11.689595 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:11.689595 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:11.689595 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:11.689595 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:11.689595 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:11.689595 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:11.689595 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:00:11.690997 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:00:11.690997 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:00:11.690997 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:00:11.690997 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 333/672] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 12:00:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:11.690997 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d248f263-8229-45c1-aab4-02da0f4e0303 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:11.703795 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 340/673] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 9 12:00:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 12:00:11.705098 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0b0e6f-c516-457e-88cb-6316ee6c2aa4 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:11.754670 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:11.803335 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d248f263-8229-45c1-aab4-02da0f4e0303 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:11.829716 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0b0e6f-c516-457e-88cb-6316ee6c2aa4 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:11.838417 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:11.850002 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0998866d-79cf-4b76-b46d-91467fc28930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:11.851094 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0998866d-79cf-4b76-b46d-91467fc28930 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:11.851810 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0998866d-79cf-4b76-b46d-91467fc28930 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:11.852269 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0998866d-79cf-4b76-b46d-91467fc28930 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:11.861425 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0998866d-79cf-4b76-b46d-91467fc28930 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:11.864116 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0998866d-79cf-4b76-b46d-91467fc28930 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:11.870015 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36aca61f-8fc6-4500-a468-bee8988c8f15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:11.870705 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aca61f-8fc6-4500-a468-bee8988c8f15 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:11.871268 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aca61f-8fc6-4500-a468-bee8988c8f15 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:11.871611 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aca61f-8fc6-4500-a468-bee8988c8f15 None admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:11.872647 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36aca61f-8fc6-4500-a468-bee8988c8f15 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:11.873038 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36aca61f-8fc6-4500-a468-bee8988c8f15 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:11.874460 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36aca61f-8fc6-4500-a468-bee8988c8f15 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:11.878680 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-0998866d-79cf-4b76-b46d-91467fc28930 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:00:11.878680 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:11.878680 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:11.878680 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:11.878680 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:11.878680 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:11.878680 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:11.878680 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:11.878680 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:11.878680 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:11.878680 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:11.878680 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:11.878680 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 12:00:11.878680 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 12:00:11.878680 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 12:00:11.880200 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 12:00:11.880200 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:11.880200 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:11.880200 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:11.880200 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:11.880200 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:11.880200 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:11.880200 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:11.880200 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:11.880200 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:11.880200 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:11.880200 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:11.880200 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:11.880200 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:11.880200 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:11.880200 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:11.880200 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:11.881104 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 12:00:11.881104 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 12:00:11.881104 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 12:00:11.881104 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 12:00:11.881104 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 12:00:11.881104 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 12:00:11.881104 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:00:11.881104 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:00:11.884063 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 334/674] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 9 12:00:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:11.885537 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0998866d-79cf-4b76-b46d-91467fc28930 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:11.888484 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 341/675] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 9 12:00:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:11.889493 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36aca61f-8fc6-4500-a468-bee8988c8f15 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:12.003554 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0998866d-79cf-4b76-b46d-91467fc28930 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:12.008620 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36aca61f-8fc6-4500-a468-bee8988c8f15 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:12.021236 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:12.040686 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a945aa79-c125-4591-bd39-0395df5d23c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:12.041458 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a945aa79-c125-4591-bd39-0395df5d23c3 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:12.041771 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a945aa79-c125-4591-bd39-0395df5d23c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:12.042104 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a945aa79-c125-4591-bd39-0395df5d23c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:12.049984 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebdb8d6-fd70-440a-b598-82ff1712a321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:12.050970 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebdb8d6-fd70-440a-b598-82ff1712a321 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:12.051430 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebdb8d6-fd70-440a-b598-82ff1712a321 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:12.051921 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebdb8d6-fd70-440a-b598-82ff1712a321 None admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:12.058636 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a945aa79-c125-4591-bd39-0395df5d23c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:12.058636 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a945aa79-c125-4591-bd39-0395df5d23c3 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:12.067162 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebdb8d6-fd70-440a-b598-82ff1712a321 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:12.067719 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebdb8d6-fd70-440a-b598-82ff1712a321 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:12.069673 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebdb8d6-fd70-440a-b598-82ff1712a321 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:12.085307 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-a945aa79-c125-4591-bd39-0395df5d23c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:00:12.085307 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:12.085307 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:12.085307 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:12.085307 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:12.085307 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:12.085307 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:12.085307 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:12.085307 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:12.085307 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:12.085307 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:12.085307 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:12.085307 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:00:12.085307 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:00:12.085307 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:00:12.085307 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:00:12.086318 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:12.086318 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:12.086318 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:12.086318 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:12.086318 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:12.086318 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:12.086318 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:12.086318 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:12.086318 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:12.086318 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:12.086318 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:12.086318 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:12.086318 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:12.086318 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:12.086318 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:12.086318 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:12.086318 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:00:12.087274 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:00:12.087274 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:00:12.087274 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:00:12.087458 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 342/677] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Sep 9 12:00:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:12.087956 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebdb8d6-fd70-440a-b598-82ff1712a321 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:12.090938 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 335/677] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 12:00:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:12.091939 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a945aa79-c125-4591-bd39-0395df5d23c3 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:12.126598 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:12.206383 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebdb8d6-fd70-440a-b598-82ff1712a321 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:12.228407 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a945aa79-c125-4591-bd39-0395df5d23c3 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:12.248053 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71f872d-3258-4ae6-aa3a-f8021a590bb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:12.249069 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71f872d-3258-4ae6-aa3a-f8021a590bb7 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:12.249554 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71f872d-3258-4ae6-aa3a-f8021a590bb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:12.250004 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71f872d-3258-4ae6-aa3a-f8021a590bb7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:12.266063 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71f872d-3258-4ae6-aa3a-f8021a590bb7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:12.267477 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71f872d-3258-4ae6-aa3a-f8021a590bb7 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:12.271638 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a20989-ace9-4b13-ae9c-ab51f7f6e816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:12.272520 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a20989-ace9-4b13-ae9c-ab51f7f6e816 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:12.273006 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a20989-ace9-4b13-ae9c-ab51f7f6e816 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:12.273473 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a20989-ace9-4b13-ae9c-ab51f7f6e816 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:12.274388 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a20989-ace9-4b13-ae9c-ab51f7f6e816 None admin] RBAC: Adding query filter params (role.id=8abeff2cb87d4b7fa72b27b028005877, scope.project.id=fe863fcd75f1406ebbffc5016078f73e, user.id=535c258100df40b58a5ee44b2c21d621) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:12.274940 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a20989-ace9-4b13-ae9c-ab51f7f6e816 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:12.276583 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a20989-ace9-4b13-ae9c-ab51f7f6e816 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:12.281448 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-b71f872d-3258-4ae6-aa3a-f8021a590bb7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 09 12:00:12.281448 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:12.281448 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:12.281448 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:12.281448 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:12.281448 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:12.281448 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:12.281448 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:12.281448 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:12.281448 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:12.281448 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:12.281448 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:12.281448 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 12:00:12.281448 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 12:00:12.281448 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 12:00:12.281448 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 12:00:12.282415 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:12.282415 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:12.282415 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 12:00:12.282415 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:00:12.282415 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 12:00:12.282415 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:00:12.282415 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:12.282415 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:12.282415 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:12.282415 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:12.282415 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:12.282415 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:12.282415 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:12.282415 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:12.282415 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:12.282415 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:12.282415 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:12.282415 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:12.283392 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:12.283392 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:12.283392 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 12:00:12.283392 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 12:00:12.283392 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 12:00:12.283392 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 12:00:12.283392 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 12:00:12.283392 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 12:00:12.283392 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 09 12:00:12.283392 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:00:12.283392 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 343/678] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Sep 9 12:00:12 2022] GET /identity/v3/users/swift => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:12.283986 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71f872d-3258-4ae6-aa3a-f8021a590bb7 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:12.287030 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 336/679] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Sep 9 12:00:12 2022] GET /identity/v3/role_assignments?user.id=535c258100df40b58a5ee44b2c21d621&scope.project.id=fe863fcd75f1406ebbffc5016078f73e&role.id=8abeff2cb87d4b7fa72b27b028005877 => generated 608 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:12.288706 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a20989-ace9-4b13-ae9c-ab51f7f6e816 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:12.324197 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:12.405759 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a20989-ace9-4b13-ae9c-ab51f7f6e816 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:12.406553 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:12.426683 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71f872d-3258-4ae6-aa3a-f8021a590bb7 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:12.445786 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffbc08b-15c9-4d0d-b60f-b5c1f1151156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:12.446770 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffbc08b-15c9-4d0d-b60f-b5c1f1151156 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:12.447277 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffbc08b-15c9-4d0d-b60f-b5c1f1151156 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:12.447749 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffbc08b-15c9-4d0d-b60f-b5c1f1151156 None admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:12.448677 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffbc08b-15c9-4d0d-b60f-b5c1f1151156 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:12.449215 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffbc08b-15c9-4d0d-b60f-b5c1f1151156 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:12.451080 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffbc08b-15c9-4d0d-b60f-b5c1f1151156 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:12.473470 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9d9979-7e81-4f45-9eaa-bf64430aa77b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:12.474296 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9d9979-7e81-4f45-9eaa-bf64430aa77b None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:12.474577 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9d9979-7e81-4f45-9eaa-bf64430aa77b None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:12.474848 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9d9979-7e81-4f45-9eaa-bf64430aa77b None admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:12.475527 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9d9979-7e81-4f45-9eaa-bf64430aa77b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:12.475931 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9d9979-7e81-4f45-9eaa-bf64430aa77b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:12.476137 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 337/680] 10.222.0.49 () {56 vars in 1135 bytes} [Fri Sep 9 12:00:12 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:00:12.477423 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9d9979-7e81-4f45-9eaa-bf64430aa77b None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:12.488115 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffbc08b-15c9-4d0d-b60f-b5c1f1151156 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:12.496232 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 344/681] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 9 12:00:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:12.497567 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9d9979-7e81-4f45-9eaa-bf64430aa77b None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:12.617741 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9d9979-7e81-4f45-9eaa-bf64430aa77b None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:12.638242 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:12.659071 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffbc08b-15c9-4d0d-b60f-b5c1f1151156 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:12.689454 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77bd7a7-3452-40c2-996d-57aaa1dae795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:12.690027 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77bd7a7-3452-40c2-996d-57aaa1dae795 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:12.690264 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77bd7a7-3452-40c2-996d-57aaa1dae795 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:12.690527 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77bd7a7-3452-40c2-996d-57aaa1dae795 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:12.701281 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77bd7a7-3452-40c2-996d-57aaa1dae795 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:12.702704 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0b282e-06da-4017-8992-f79ce1585134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:12.703180 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77bd7a7-3452-40c2-996d-57aaa1dae795 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:12.703959 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0b282e-06da-4017-8992-f79ce1585134 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:12.704433 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0b282e-06da-4017-8992-f79ce1585134 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:12.704876 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0b282e-06da-4017-8992-f79ce1585134 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:12.717878 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-b77bd7a7-3452-40c2-996d-57aaa1dae795 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:00:12.717878 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:12.717878 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:12.717878 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:12.717878 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:12.717878 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:12.717878 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:12.717878 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:12.717878 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:12.717878 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:12.717878 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:12.717878 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:12.717878 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 12:00:12.717878 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 12:00:12.717878 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 12:00:12.718830 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 12:00:12.718830 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:12.718830 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:12.718830 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:12.718830 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:12.718830 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:12.718830 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:12.718830 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:12.718830 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:12.718830 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:12.718830 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:12.718830 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:12.718830 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:12.718830 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:12.718830 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:12.718830 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:12.718830 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:12.719770 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 12:00:12.719770 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 12:00:12.719770 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 12:00:12.719770 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 12:00:12.719770 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 12:00:12.719770 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 12:00:12.719770 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:00:12.719770 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:00:12.723004 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 345/682] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 9 12:00:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:12.726524 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77bd7a7-3452-40c2-996d-57aaa1dae795 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:12.737632 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:12.796864 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0b282e-06da-4017-8992-f79ce1585134 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:12.798653 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0b282e-06da-4017-8992-f79ce1585134 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:12.864194 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77bd7a7-3452-40c2-996d-57aaa1dae795 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:12.918133 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9091cefd-628b-4d1e-93ec-cd0bf4b5c809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:12.918663 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9091cefd-628b-4d1e-93ec-cd0bf4b5c809 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:12.919120 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9091cefd-628b-4d1e-93ec-cd0bf4b5c809 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:12.919394 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9091cefd-628b-4d1e-93ec-cd0bf4b5c809 None admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:12.921562 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9091cefd-628b-4d1e-93ec-cd0bf4b5c809 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:12.921869 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9091cefd-628b-4d1e-93ec-cd0bf4b5c809 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:12.925278 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9091cefd-628b-4d1e-93ec-cd0bf4b5c809 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:12.942396 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:12.950588 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 346/683] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Sep 9 12:00:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:12.957352 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9091cefd-628b-4d1e-93ec-cd0bf4b5c809 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:12.966617 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 338/684] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 9 12:00:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 12:00:12.967997 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0b282e-06da-4017-8992-f79ce1585134 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:13.065388 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:13.134314 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9091cefd-628b-4d1e-93ec-cd0bf4b5c809 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:13.189233 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e44feb-6e8b-48d5-bb57-a2ae316bf810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:13.190005 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e44feb-6e8b-48d5-bb57-a2ae316bf810 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:13.190281 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e44feb-6e8b-48d5-bb57-a2ae316bf810 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:13.190552 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e44feb-6e8b-48d5-bb57-a2ae316bf810 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:13.193086 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e44feb-6e8b-48d5-bb57-a2ae316bf810 None admin] RBAC: Adding query filter params (role.id=8abeff2cb87d4b7fa72b27b028005877, scope.project.id=fe863fcd75f1406ebbffc5016078f73e, user.id=992de6f86b42495dafe73cb2c393fb4e) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:13.193332 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e44feb-6e8b-48d5-bb57-a2ae316bf810 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:13.194778 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e44feb-6e8b-48d5-bb57-a2ae316bf810 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:13.208102 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 347/685] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Sep 9 12:00:12 2022] GET /identity/v3/role_assignments?user.id=992de6f86b42495dafe73cb2c393fb4e&scope.project.id=fe863fcd75f1406ebbffc5016078f73e&role.id=8abeff2cb87d4b7fa72b27b028005877 => generated 608 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:13.230602 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0b282e-06da-4017-8992-f79ce1585134 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:13.298372 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4494dc-0e6b-49ab-bbb5-809475195db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:13.299050 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4494dc-0e6b-49ab-bbb5-809475195db0 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:13.299328 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4494dc-0e6b-49ab-bbb5-809475195db0 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:13.299597 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4494dc-0e6b-49ab-bbb5-809475195db0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:13.313396 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4494dc-0e6b-49ab-bbb5-809475195db0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:13.315170 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4494dc-0e6b-49ab-bbb5-809475195db0 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:13.335339 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-4e4494dc-0e6b-49ab-bbb5-809475195db0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:00:13.335339 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:13.335339 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:13.335339 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:13.335339 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:13.335339 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:13.335339 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:13.335339 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:13.335339 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:13.335339 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:13.335339 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:13.335339 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:13.335339 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:00:13.335339 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:00:13.335339 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:00:13.335339 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:00:13.336370 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:13.336370 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:13.336370 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:13.336370 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:13.336370 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:13.336370 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:13.336370 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:13.336370 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:13.336370 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:13.336370 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:13.336370 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:13.336370 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:13.336370 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:13.336370 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:13.336370 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:13.336370 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:13.336370 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:00:13.337919 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:00:13.337919 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:00:13.337919 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:00:13.337919 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 339/686] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 12:00:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 369 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 12:00:13.342879 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e44feb-6e8b-48d5-bb57-a2ae316bf810 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:13.349193 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92dd837-8f5c-4441-a479-f59be0d2a33a None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:13.349451 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92dd837-8f5c-4441-a479-f59be0d2a33a None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:13.349748 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92dd837-8f5c-4441-a479-f59be0d2a33a None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:13.351363 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 340/687] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:00:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 12:00:13.358105 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227b0197-d66a-4547-b72e-0b65d038f815 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:13.358360 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227b0197-d66a-4547-b72e-0b65d038f815 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:13.358713 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227b0197-d66a-4547-b72e-0b65d038f815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:13.379061 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-227b0197-d66a-4547-b72e-0b65d038f815 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:00:13.380579 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-227b0197-d66a-4547-b72e-0b65d038f815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:13.443508 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:13.533616 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 341/688] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:00:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 12:00:13.562363 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e44feb-6e8b-48d5-bb57-a2ae316bf810 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:13.598420 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:13.610726 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7201833-4a47-46f9-a6be-e86f5d2d3f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:13.611395 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7201833-4a47-46f9-a6be-e86f5d2d3f0b None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:13.611672 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7201833-4a47-46f9-a6be-e86f5d2d3f0b None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:13.611944 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7201833-4a47-46f9-a6be-e86f5d2d3f0b None admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:13.612600 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7201833-4a47-46f9-a6be-e86f5d2d3f0b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:13.612924 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7201833-4a47-46f9-a6be-e86f5d2d3f0b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:13.614825 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7201833-4a47-46f9-a6be-e86f5d2d3f0b None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:13.630432 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 348/689] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 9 12:00:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:13.639946 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227b0197-d66a-4547-b72e-0b65d038f815 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:13.856617 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227b0197-d66a-4547-b72e-0b65d038f815 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:13.888177 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7201833-4a47-46f9-a6be-e86f5d2d3f0b None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:13.910295 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59eeec5-3d58-4b2d-9615-26e545fb78a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:13.911477 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59eeec5-3d58-4b2d-9615-26e545fb78a3 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:13.911988 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59eeec5-3d58-4b2d-9615-26e545fb78a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:13.912463 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59eeec5-3d58-4b2d-9615-26e545fb78a3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:13.925211 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59eeec5-3d58-4b2d-9615-26e545fb78a3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:13.929849 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59eeec5-3d58-4b2d-9615-26e545fb78a3 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:13.951331 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-f59eeec5-3d58-4b2d-9615-26e545fb78a3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:00:13.951331 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:13.951331 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:13.951331 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:13.951331 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:13.951331 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:13.951331 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:13.951331 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:13.951331 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:13.951331 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:13.951331 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:13.951331 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:13.951331 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 12:00:13.951331 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 12:00:13.951331 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 12:00:13.952297 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 12:00:13.952297 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:13.952297 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:13.952297 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:13.952297 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:13.952297 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:13.952297 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:13.952297 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:13.952297 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:13.952297 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:13.952297 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:13.952297 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:13.952297 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:13.952297 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:13.952297 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:13.952297 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:13.952297 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:13.953221 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 12:00:13.953221 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 12:00:13.953221 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 12:00:13.953221 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 12:00:13.953221 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 12:00:13.953221 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 12:00:13.953221 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:00:13.953221 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:00:13.960858 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 342/690] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 9 12:00:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 12:00:13.962387 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59eeec5-3d58-4b2d-9615-26e545fb78a3 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:14.023591 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:14.115801 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7201833-4a47-46f9-a6be-e86f5d2d3f0b None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:14.174551 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a5ca60-10bb-40df-be45-08ecb01926b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:14.175250 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a5ca60-10bb-40df-be45-08ecb01926b5 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:14.176075 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a5ca60-10bb-40df-be45-08ecb01926b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:14.178938 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a5ca60-10bb-40df-be45-08ecb01926b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:14.196314 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a5ca60-10bb-40df-be45-08ecb01926b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:14.197804 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a5ca60-10bb-40df-be45-08ecb01926b5 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:14.215276 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-a8a5ca60-10bb-40df-be45-08ecb01926b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:00:14.215276 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:14.215276 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:14.215276 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:14.215276 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:14.215276 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:14.215276 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:14.215276 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:14.215276 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:14.215276 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:14.215276 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:14.215276 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:14.215276 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:00:14.215276 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:00:14.215276 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:00:14.215276 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:00:14.216272 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:14.216272 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:14.216272 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:14.216272 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:14.216272 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:14.216272 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:14.216272 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:14.216272 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:14.216272 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:14.216272 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:14.216272 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:14.216272 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:14.216272 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:14.216272 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:14.216272 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:14.216272 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:14.216272 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:00:14.217251 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:00:14.217251 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:00:14.217251 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:00:14.217251 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 349/691] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 12:00:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:14.223298 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a5ca60-10bb-40df-be45-08ecb01926b5 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:14.242629 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59eeec5-3d58-4b2d-9615-26e545fb78a3 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:14.299562 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:14.305261 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5427a988-0f07-4b29-9004-1e75a31999c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:14.306384 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5427a988-0f07-4b29-9004-1e75a31999c9 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:14.306664 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5427a988-0f07-4b29-9004-1e75a31999c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:14.307247 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5427a988-0f07-4b29-9004-1e75a31999c9 None admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:14.307946 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5427a988-0f07-4b29-9004-1e75a31999c9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:14.308488 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5427a988-0f07-4b29-9004-1e75a31999c9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:14.310315 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5427a988-0f07-4b29-9004-1e75a31999c9 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:14.328676 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 343/692] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Sep 9 12:00:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 366 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:14.330333 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28601c25-9597-4997-8091-8ae56a3d5742 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:14.330620 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28601c25-9597-4997-8091-8ae56a3d5742 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:14.330892 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28601c25-9597-4997-8091-8ae56a3d5742 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:14.338459 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 344/693] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:00:14 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:00:14.339277 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28601c25-9597-4997-8091-8ae56a3d5742 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:14.363834 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a5ca60-10bb-40df-be45-08ecb01926b5 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:14.400984 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b8c848-17f4-456f-a002-fbaf659cdeeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:14.401618 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b8c848-17f4-456f-a002-fbaf659cdeeb None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:14.401974 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b8c848-17f4-456f-a002-fbaf659cdeeb None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:14.402387 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b8c848-17f4-456f-a002-fbaf659cdeeb None admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:14.403171 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b8c848-17f4-456f-a002-fbaf659cdeeb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:14.403581 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b8c848-17f4-456f-a002-fbaf659cdeeb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:14.405259 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b8c848-17f4-456f-a002-fbaf659cdeeb None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:14.420424 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 350/694] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 9 12:00:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:14.421812 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b246bef-46e9-4a8d-8d5b-82c7caa03221 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:14.422095 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b246bef-46e9-4a8d-8d5b-82c7caa03221 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:14.422366 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b246bef-46e9-4a8d-8d5b-82c7caa03221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:14.446012 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-8b246bef-46e9-4a8d-8d5b-82c7caa03221 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:00:14.447317 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-8b246bef-46e9-4a8d-8d5b-82c7caa03221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:14.458291 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:14.472530 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28601c25-9597-4997-8091-8ae56a3d5742 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:14.519932 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8be840-ecbf-474a-8d62-ffc6d8de2fc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:14.520895 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8be840-ecbf-474a-8d62-ffc6d8de2fc3 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:14.521178 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8be840-ecbf-474a-8d62-ffc6d8de2fc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:14.521447 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8be840-ecbf-474a-8d62-ffc6d8de2fc3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:14.522169 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8be840-ecbf-474a-8d62-ffc6d8de2fc3 None admin] RBAC: Adding query filter params (role.id=8abeff2cb87d4b7fa72b27b028005877, scope.project.id=fe863fcd75f1406ebbffc5016078f73e, user.id=92e652ba827849b5a3fc1213d04a1b76) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:14.522478 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8be840-ecbf-474a-8d62-ffc6d8de2fc3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:14.524047 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8be840-ecbf-474a-8d62-ffc6d8de2fc3 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:14.524826 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 351/695] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:00:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 12:00:14.525696 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b246bef-46e9-4a8d-8d5b-82c7caa03221 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:14.541333 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 345/696] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Sep 9 12:00:14 2022] GET /identity/v3/role_assignments?user.id=92e652ba827849b5a3fc1213d04a1b76&scope.project.id=fe863fcd75f1406ebbffc5016078f73e&role.id=8abeff2cb87d4b7fa72b27b028005877 => generated 608 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:14.569666 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:14.658934 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b246bef-46e9-4a8d-8d5b-82c7caa03221 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:14.710114 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea479683-d724-49cc-b093-d8352c642a36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:14.711067 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea479683-d724-49cc-b093-d8352c642a36 None admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:14.711496 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea479683-d724-49cc-b093-d8352c642a36 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:14.711981 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea479683-d724-49cc-b093-d8352c642a36 None admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:14.712871 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea479683-d724-49cc-b093-d8352c642a36 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:14.714555 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea479683-d724-49cc-b093-d8352c642a36 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:14.752649 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 352/697] 10.222.0.49 () {60 vars in 1132 bytes} [Fri Sep 9 12:00:14 2022] POST /identity/v3/users => generated 254 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:00:14.936344 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8be840-ecbf-474a-8d62-ffc6d8de2fc3 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:15.054726 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8be840-ecbf-474a-8d62-ffc6d8de2fc3 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:15.098281 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f9eb6d-3788-40d1-a582-52e6e3859948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:15.099411 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f9eb6d-3788-40d1-a582-52e6e3859948 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:15.099917 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f9eb6d-3788-40d1-a582-52e6e3859948 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:15.100398 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f9eb6d-3788-40d1-a582-52e6e3859948 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:15.113925 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f9eb6d-3788-40d1-a582-52e6e3859948 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:00:15.119621 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-48f9eb6d-3788-40d1-a582-52e6e3859948 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 12:00:15.119621 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:15.119621 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:15.119621 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:15.119621 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:15.119621 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:15.119621 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:15.119621 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:15.119621 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:15.119621 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:15.119621 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:15.119621 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:15.119621 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:00:15.119621 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:00:15.119621 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:00:15.119621 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:00:15.120630 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:00:15.120630 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:00:15.120630 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:00:15.120630 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:00:15.120630 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:15.120630 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:15.120630 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:15.120630 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:15.120630 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:15.120630 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:15.120630 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:15.120630 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:15.120630 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:15.120630 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:15.120630 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:15.120630 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:15.121477 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:15.121477 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:15.121477 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:15.121477 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:15.121477 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:00:15.121477 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:00:15.121477 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:00:15.121477 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:00:15.121477 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:00:15.121477 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:00:15.121477 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 12:00:15.121477 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:00:15.124397 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 346/698] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Sep 9 12:00:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:15.131086 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea479683-d724-49cc-b093-d8352c642a36 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:15.235275 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67493f26-02ef-47e5-9991-0a85accf14be None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:15.235924 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67493f26-02ef-47e5-9991-0a85accf14be None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:15.236405 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67493f26-02ef-47e5-9991-0a85accf14be None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:15.238383 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 347/699] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:00:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:00:15.238959 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:15.244136 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836ee20e-5444-456d-96be-83627b15bd61 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:15.244694 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836ee20e-5444-456d-96be-83627b15bd61 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:15.245177 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836ee20e-5444-456d-96be-83627b15bd61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:15.257469 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea479683-d724-49cc-b093-d8352c642a36 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:15.270487 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-836ee20e-5444-456d-96be-83627b15bd61 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:00:15.272118 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-836ee20e-5444-456d-96be-83627b15bd61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:15.287302 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:15.326943 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13e4391-8b39-413b-b5a3-529f3dd7bc75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:15.327574 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13e4391-8b39-413b-b5a3-529f3dd7bc75 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:15.327859 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13e4391-8b39-413b-b5a3-529f3dd7bc75 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:15.328130 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13e4391-8b39-413b-b5a3-529f3dd7bc75 None admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:15.328801 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13e4391-8b39-413b-b5a3-529f3dd7bc75 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:15.329152 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13e4391-8b39-413b-b5a3-529f3dd7bc75 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:15.330467 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13e4391-8b39-413b-b5a3-529f3dd7bc75 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:15.341060 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 353/700] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 9 12:00:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:15.349138 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13e4391-8b39-413b-b5a3-529f3dd7bc75 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:15.366476 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 348/701] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:00:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 12:00:15.464687 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13e4391-8b39-413b-b5a3-529f3dd7bc75 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:15.500883 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f708755-4906-4896-a30b-054dbf389fd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:15.501515 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f708755-4906-4896-a30b-054dbf389fd2 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:15.501790 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f708755-4906-4896-a30b-054dbf389fd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:15.502057 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f708755-4906-4896-a30b-054dbf389fd2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:15.514719 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f708755-4906-4896-a30b-054dbf389fd2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:15.516276 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f708755-4906-4896-a30b-054dbf389fd2 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:15.532195 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-1f708755-4906-4896-a30b-054dbf389fd2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:00:15.532195 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:15.532195 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:15.532195 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:15.532195 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:15.532195 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:15.532195 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:15.532195 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:15.532195 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:15.532195 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:15.532195 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:15.532195 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:15.532195 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:00:15.532195 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:00:15.532195 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:00:15.532195 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:00:15.533252 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:15.533252 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:15.533252 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:15.533252 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:15.533252 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:15.533252 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:15.533252 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:15.533252 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:15.533252 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:15.533252 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:15.533252 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:15.533252 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:15.533252 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:15.533252 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:15.533252 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:15.533252 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:15.533252 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:00:15.534174 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:00:15.534174 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:00:15.534174 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:00:15.534174 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 354/702] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 12:00:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:15.542168 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836ee20e-5444-456d-96be-83627b15bd61 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:15.654838 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836ee20e-5444-456d-96be-83627b15bd61 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:15.691818 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7773d99a-ea14-4256-b332-f0ade5035098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:15.692473 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7773d99a-ea14-4256-b332-f0ade5035098 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:15.692749 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7773d99a-ea14-4256-b332-f0ade5035098 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:15.693017 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7773d99a-ea14-4256-b332-f0ade5035098 None admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:15.693651 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7773d99a-ea14-4256-b332-f0ade5035098 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:15.693949 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7773d99a-ea14-4256-b332-f0ade5035098 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:15.695336 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7773d99a-ea14-4256-b332-f0ade5035098 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:15.708468 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 349/703] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 9 12:00:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:00:15.713976 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f708755-4906-4896-a30b-054dbf389fd2 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:15.716836 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:15.827226 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f708755-4906-4896-a30b-054dbf389fd2 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:15.866170 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b14aa0-30b0-49ba-a8f3-ba5f5d3a188b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:15.867168 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b14aa0-30b0-49ba-a8f3-ba5f5d3a188b None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:15.867619 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b14aa0-30b0-49ba-a8f3-ba5f5d3a188b None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:15.868096 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b14aa0-30b0-49ba-a8f3-ba5f5d3a188b None admin] PATH_INFO: `/v3/users/nova` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:15.876367 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b14aa0-30b0-49ba-a8f3-ba5f5d3a188b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:15.878023 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b14aa0-30b0-49ba-a8f3-ba5f5d3a188b None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:15.888003 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7773d99a-ea14-4256-b332-f0ade5035098 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:15.898644 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:15.901334 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-d1b14aa0-30b0-49ba-a8f3-ba5f5d3a188b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 12:00:15.901334 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:15.901334 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:15.901334 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:15.901334 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:15.901334 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:15.901334 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:15.901334 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:15.901334 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:15.901334 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:15.901334 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:15.901334 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:15.901334 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 12:00:15.901334 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 12:00:15.901334 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 12:00:15.901334 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 12:00:15.902414 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:15.902414 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:15.902414 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 12:00:15.902414 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:00:15.902414 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 12:00:15.902414 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:00:15.902414 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:15.902414 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:15.902414 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:15.902414 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:15.902414 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:15.902414 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:15.902414 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:15.902414 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:15.902414 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:15.902414 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:15.902414 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:15.902414 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:15.903398 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:15.903398 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:15.903398 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 12:00:15.903398 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 12:00:15.903398 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 12:00:15.903398 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 12:00:15.903398 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 12:00:15.903398 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 12:00:15.903398 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 12:00:15.903398 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:00:15.909155 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 355/704] 10.222.0.49 () {56 vars in 1093 bytes} [Fri Sep 9 12:00:15 2022] GET /identity/v3/users/nova => generated 82 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:15.912186 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b14aa0-30b0-49ba-a8f3-ba5f5d3a188b None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:16.013790 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:16.060143 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b14aa0-30b0-49ba-a8f3-ba5f5d3a188b None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:16.096711 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7773d99a-ea14-4256-b332-f0ade5035098 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:16.103816 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ad5779-c9ac-4e29-8b7f-60231fc4424a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:16.104438 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ad5779-c9ac-4e29-8b7f-60231fc4424a None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:16.104714 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ad5779-c9ac-4e29-8b7f-60231fc4424a None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:16.104981 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ad5779-c9ac-4e29-8b7f-60231fc4424a None admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:16.105720 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ad5779-c9ac-4e29-8b7f-60231fc4424a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:16.106018 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ad5779-c9ac-4e29-8b7f-60231fc4424a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:16.107586 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ad5779-c9ac-4e29-8b7f-60231fc4424a None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:16.131361 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 356/705] 10.222.0.49 () {56 vars in 1133 bytes} [Fri Sep 9 12:00:15 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:00:16.137420 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ad5779-c9ac-4e29-8b7f-60231fc4424a None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:16.172221 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b06ac80-f3d3-451f-ac5a-7a159a10ba0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:16.173248 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b06ac80-f3d3-451f-ac5a-7a159a10ba0c None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:16.173844 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b06ac80-f3d3-451f-ac5a-7a159a10ba0c None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:16.174342 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b06ac80-f3d3-451f-ac5a-7a159a10ba0c None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:16.175163 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b06ac80-f3d3-451f-ac5a-7a159a10ba0c None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:16.177495 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b06ac80-f3d3-451f-ac5a-7a159a10ba0c None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:16.200453 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-6b06ac80-f3d3-451f-ac5a-7a159a10ba0c None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 12:00:16.200453 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:16.200453 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 12:00:16.200453 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 12:00:16.200453 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 12:00:16.200453 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 12:00:16.200453 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 12:00:16.200453 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 12:00:16.200453 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 12:00:16.200453 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:00:16.200453 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 12:00:16.200453 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:00:16.200453 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:16.200453 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:16.200453 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:16.200453 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:16.200453 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:16.201570 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:16.201570 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:16.201570 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:16.201570 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:16.201570 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:16.201570 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:16.201570 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 12:00:16.201570 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 12:00:16.201570 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 12:00:16.201570 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 12:00:16.201570 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:16.201570 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:16.201570 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:16.201570 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:16.201570 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:16.201570 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:16.202442 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:16.202442 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:16.202442 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:16.202442 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:16.202442 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:16.202442 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:16.202442 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:16.202442 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:16.202442 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:16.202442 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:16.202442 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 12:00:16.202442 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 12:00:16.202442 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 12:00:16.202442 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:00:16.210925 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 350/706] 10.222.0.49 () {56 vars in 1129 bytes} [Fri Sep 9 12:00:15 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 12:00:16.210925 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae93b65-3178-4dbc-8d29-2c6feab1c35b None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:16.211144 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae93b65-3178-4dbc-8d29-2c6feab1c35b None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:16.211383 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae93b65-3178-4dbc-8d29-2c6feab1c35b None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:16.213733 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 351/707] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:00:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:00:16.214454 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae93b65-3178-4dbc-8d29-2c6feab1c35b None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:16.247709 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ad5779-c9ac-4e29-8b7f-60231fc4424a None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:16.270815 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:16.286086 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fcdbae-69ec-47f2-b24c-c78f685a748d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:16.286970 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fcdbae-69ec-47f2-b24c-c78f685a748d None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:16.287325 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fcdbae-69ec-47f2-b24c-c78f685a748d None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:16.287912 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fcdbae-69ec-47f2-b24c-c78f685a748d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:16.295035 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:16.332200 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae93b65-3178-4dbc-8d29-2c6feab1c35b None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:16.374344 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fcdbae-69ec-47f2-b24c-c78f685a748d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:16.375844 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fcdbae-69ec-47f2-b24c-c78f685a748d None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:16.379148 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b835f2-8ad7-42cd-ac03-dffba12a885b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:16.381048 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b835f2-8ad7-42cd-ac03-dffba12a885b None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:16.381579 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b835f2-8ad7-42cd-ac03-dffba12a885b None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:16.382046 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b835f2-8ad7-42cd-ac03-dffba12a885b None admin] PATH_INFO: `/v3/services` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:16.382940 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b835f2-8ad7-42cd-ac03-dffba12a885b None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:16.383479 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b835f2-8ad7-42cd-ac03-dffba12a885b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:16.385444 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b835f2-8ad7-42cd-ac03-dffba12a885b None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:16.391793 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 352/708] 10.222.0.49 () {56 vars in 1124 bytes} [Fri Sep 9 12:00:16 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:16.393783 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272fcf38-b7b3-44de-8f76-8c9e2cc4c5d6 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:16.394352 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272fcf38-b7b3-44de-8f76-8c9e2cc4c5d6 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:16.395422 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272fcf38-b7b3-44de-8f76-8c9e2cc4c5d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:16.419055 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-272fcf38-b7b3-44de-8f76-8c9e2cc4c5d6 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:00:16.420946 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-272fcf38-b7b3-44de-8f76-8c9e2cc4c5d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:16.490420 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 357/709] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 9 12:00:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 12:00:16.491593 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fcdbae-69ec-47f2-b24c-c78f685a748d None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:16.507506 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 353/710] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:00:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 12:00:16.508876 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272fcf38-b7b3-44de-8f76-8c9e2cc4c5d6 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:16.524172 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:16.580439 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:16.623358 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272fcf38-b7b3-44de-8f76-8c9e2cc4c5d6 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:16.661995 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630433fc-34a0-4a43-8ff2-d9fe3e8e6c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:16.663257 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630433fc-34a0-4a43-8ff2-d9fe3e8e6c71 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:16.663731 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630433fc-34a0-4a43-8ff2-d9fe3e8e6c71 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:16.664191 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630433fc-34a0-4a43-8ff2-d9fe3e8e6c71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:16.676370 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630433fc-34a0-4a43-8ff2-d9fe3e8e6c71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:16.678966 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630433fc-34a0-4a43-8ff2-d9fe3e8e6c71 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:16.696636 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-630433fc-34a0-4a43-8ff2-d9fe3e8e6c71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:00:16.696636 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:16.696636 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:16.696636 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:16.696636 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:16.696636 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:16.696636 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:16.696636 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:16.696636 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:16.696636 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:16.696636 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:16.696636 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:16.696636 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:00:16.696636 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:00:16.696636 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:00:16.696636 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:00:16.697615 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:16.697615 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:16.697615 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:16.697615 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:16.697615 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:16.697615 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:16.697615 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:16.697615 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:16.697615 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:16.697615 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:16.697615 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:16.697615 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:16.697615 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:16.697615 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:16.697615 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:16.697615 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:16.697615 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:00:16.698501 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:00:16.698501 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:00:16.698501 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:00:16.700568 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 354/711] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 12:00:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:16.703818 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fcdbae-69ec-47f2-b24c-c78f685a748d None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:16.706338 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630433fc-34a0-4a43-8ff2-d9fe3e8e6c71 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:16.760217 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d308fa91-83dd-45d8-ae1e-59ce854c5aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:16.762121 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d308fa91-83dd-45d8-ae1e-59ce854c5aa8 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:16.763686 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d308fa91-83dd-45d8-ae1e-59ce854c5aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:16.764180 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d308fa91-83dd-45d8-ae1e-59ce854c5aa8 None admin] PATH_INFO: `/v3/services` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:16.765737 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d308fa91-83dd-45d8-ae1e-59ce854c5aa8 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:16.767604 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d308fa91-83dd-45d8-ae1e-59ce854c5aa8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:16.769281 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d308fa91-83dd-45d8-ae1e-59ce854c5aa8 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:16.781545 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 358/712] 10.222.0.49 () {56 vars in 1124 bytes} [Fri Sep 9 12:00:16 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:16.856111 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630433fc-34a0-4a43-8ff2-d9fe3e8e6c71 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:16.874956 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:16.885460 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d308fa91-83dd-45d8-ae1e-59ce854c5aa8 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:16.895010 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acc4764-e6c9-4fbb-a226-6464c1b0ebba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:16.895864 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acc4764-e6c9-4fbb-a226-6464c1b0ebba None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:16.896137 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acc4764-e6c9-4fbb-a226-6464c1b0ebba None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:16.896404 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acc4764-e6c9-4fbb-a226-6464c1b0ebba None admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:16.897062 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acc4764-e6c9-4fbb-a226-6464c1b0ebba None admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:16.897362 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acc4764-e6c9-4fbb-a226-6464c1b0ebba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:16.898791 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acc4764-e6c9-4fbb-a226-6464c1b0ebba None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:16.913229 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 355/713] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 9 12:00:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:16.920083 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acc4764-e6c9-4fbb-a226-6464c1b0ebba None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:17.050248 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acc4764-e6c9-4fbb-a226-6464c1b0ebba None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:17.056283 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:17.076960 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d308fa91-83dd-45d8-ae1e-59ce854c5aa8 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:17.098120 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28945de5-bc1f-4531-831d-707c76071c58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:17.098859 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28945de5-bc1f-4531-831d-707c76071c58 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:17.099058 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28945de5-bc1f-4531-831d-707c76071c58 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:17.099315 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28945de5-bc1f-4531-831d-707c76071c58 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:17.108171 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28945de5-bc1f-4531-831d-707c76071c58 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:17.109729 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28945de5-bc1f-4531-831d-707c76071c58 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:17.120891 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e201934e-7325-4885-86f1-19a77b6174f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:17.121850 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e201934e-7325-4885-86f1-19a77b6174f3 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:17.122334 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e201934e-7325-4885-86f1-19a77b6174f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:17.122824 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e201934e-7325-4885-86f1-19a77b6174f3 None admin] PATH_INFO: `/v3/services/network` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:17.123633 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e201934e-7325-4885-86f1-19a77b6174f3 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:17.124594 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-28945de5-bc1f-4531-831d-707c76071c58 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:00:17.124594 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:17.124594 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:17.124594 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:17.124594 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:17.124594 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:17.124594 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:17.124594 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:17.124594 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:17.124594 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:17.124594 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:17.124594 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:17.124594 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 12:00:17.124594 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 12:00:17.124594 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 12:00:17.125444 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 12:00:17.125444 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:17.125444 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:17.125444 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:17.125444 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:17.125444 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:17.125444 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:17.125444 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:17.125444 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:17.125444 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:17.125444 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:17.125444 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:17.125444 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:17.125444 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:17.125444 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:17.125444 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:17.125444 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:17.126354 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 12:00:17.126354 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 12:00:17.126354 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 12:00:17.126354 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 12:00:17.126354 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 12:00:17.126354 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 12:00:17.126354 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:00:17.126354 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:00:17.128087 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e201934e-7325-4885-86f1-19a77b6174f3 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:17.129844 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 356/714] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 9 12:00:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:17.132739 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28945de5-bc1f-4531-831d-707c76071c58 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:17.143152 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-e201934e-7325-4885-86f1-19a77b6174f3 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 12:00:17.143152 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:17.143152 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 12:00:17.143152 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 12:00:17.143152 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 12:00:17.143152 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 12:00:17.143152 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 12:00:17.143152 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 12:00:17.143152 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 12:00:17.143152 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:00:17.143152 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 12:00:17.143152 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:00:17.143152 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:17.143152 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:17.143152 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:17.143152 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:17.143152 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:17.144542 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:17.144542 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:17.144542 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:17.144542 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:17.144542 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:17.144542 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:17.144542 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 12:00:17.144542 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 12:00:17.144542 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 12:00:17.144542 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 12:00:17.144542 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:17.144542 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:17.144542 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:17.144542 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:17.144542 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:17.144542 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:17.145397 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:17.145397 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:17.145397 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:17.145397 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:17.145397 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:17.145397 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:17.145397 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:17.145397 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:17.145397 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:17.145397 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:17.145397 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 12:00:17.145397 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 12:00:17.145397 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 12:00:17.145397 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:00:17.148365 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 359/715] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 9 12:00:16 2022] GET /identity/v3/services/network => generated 88 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:17.157231 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e201934e-7325-4885-86f1-19a77b6174f3 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:17.179907 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:17.303393 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28945de5-bc1f-4531-831d-707c76071c58 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:17.355689 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e201934e-7325-4885-86f1-19a77b6174f3 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:17.376642 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086723fe-b88d-4876-a270-841f40284cff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:17.377248 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086723fe-b88d-4876-a270-841f40284cff None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:17.377521 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086723fe-b88d-4876-a270-841f40284cff None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:17.377786 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086723fe-b88d-4876-a270-841f40284cff None admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:17.379090 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086723fe-b88d-4876-a270-841f40284cff None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:17.379579 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086723fe-b88d-4876-a270-841f40284cff None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:17.381297 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086723fe-b88d-4876-a270-841f40284cff None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:17.396104 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 357/716] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Sep 9 12:00:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:00:17.402735 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086723fe-b88d-4876-a270-841f40284cff None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:17.416448 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24e27b3-7ec8-4d68-9684-ea49cedb0426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:17.417070 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24e27b3-7ec8-4d68-9684-ea49cedb0426 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:17.417347 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24e27b3-7ec8-4d68-9684-ea49cedb0426 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:17.417616 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24e27b3-7ec8-4d68-9684-ea49cedb0426 None admin] PATH_INFO: `/v3/services` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:17.418247 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24e27b3-7ec8-4d68-9684-ea49cedb0426 None admin] RBAC: Adding query filter params (name=network) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:17.418546 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24e27b3-7ec8-4d68-9684-ea49cedb0426 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:17.419967 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24e27b3-7ec8-4d68-9684-ea49cedb0426 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:17.424649 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 360/717] 10.222.0.49 () {56 vars in 1112 bytes} [Fri Sep 9 12:00:17 2022] GET /identity/v3/services?name=network => generated 124 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:17.426121 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e211e173-5047-4198-ac67-7b8cc8bbb109 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:17.426397 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e211e173-5047-4198-ac67-7b8cc8bbb109 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:17.426668 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e211e173-5047-4198-ac67-7b8cc8bbb109 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:17.428157 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 361/718] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:00:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:00:17.432947 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e211e173-5047-4198-ac67-7b8cc8bbb109 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:17.450404 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:17.521647 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:17.528220 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086723fe-b88d-4876-a270-841f40284cff None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:17.557928 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e211e173-5047-4198-ac67-7b8cc8bbb109 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:17.568782 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b193e01d-b201-42dd-85aa-70712451384f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:17.569534 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b193e01d-b201-42dd-85aa-70712451384f None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:17.569881 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b193e01d-b201-42dd-85aa-70712451384f None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:17.570219 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b193e01d-b201-42dd-85aa-70712451384f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:17.571220 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b193e01d-b201-42dd-85aa-70712451384f None admin] RBAC: Adding query filter params (role.id=3f2ca33902a04b6dbc2ef7d06d7289d3, scope.project.id=fe863fcd75f1406ebbffc5016078f73e, user.id=70bac169d5cd43e1926f310344ac11fa) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:17.571614 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b193e01d-b201-42dd-85aa-70712451384f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:17.573223 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b193e01d-b201-42dd-85aa-70712451384f None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:17.579470 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 358/719] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Sep 9 12:00:17 2022] GET /identity/v3/role_assignments?user.id=70bac169d5cd43e1926f310344ac11fa&scope.project.id=fe863fcd75f1406ebbffc5016078f73e&role.id=3f2ca33902a04b6dbc2ef7d06d7289d3 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:00:17.580848 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85413f18-a568-4efe-946d-166ca195ad7b None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:17.581195 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85413f18-a568-4efe-946d-166ca195ad7b None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:17.581589 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85413f18-a568-4efe-946d-166ca195ad7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:17.602874 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-85413f18-a568-4efe-946d-166ca195ad7b None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:00:17.603238 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c9d430-4d18-404c-aece-68de323eb62d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:17.604102 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c9d430-4d18-404c-aece-68de323eb62d None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:17.604486 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-85413f18-a568-4efe-946d-166ca195ad7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:17.604738 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c9d430-4d18-404c-aece-68de323eb62d None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:17.605177 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c9d430-4d18-404c-aece-68de323eb62d None admin] PATH_INFO: `/v3/services` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:17.606032 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c9d430-4d18-404c-aece-68de323eb62d None admin] RBAC: Adding query filter params (type=network) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:17.606516 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c9d430-4d18-404c-aece-68de323eb62d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:17.608408 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c9d430-4d18-404c-aece-68de323eb62d None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:17.615262 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 362/720] 10.222.0.49 () {56 vars in 1112 bytes} [Fri Sep 9 12:00:17 2022] GET /identity/v3/services?type=network => generated 124 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:17.717926 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 359/721] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:00:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 12:00:17.741065 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae034aa-74be-48e6-9fb7-cf01f201d052 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:17.741389 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae034aa-74be-48e6-9fb7-cf01f201d052 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:17.741621 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae034aa-74be-48e6-9fb7-cf01f201d052 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:17.743373 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 363/722] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:00:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:00:17.749329 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba7e826-eb6c-40b1-9562-9b64c52a35b5 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:17.749612 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba7e826-eb6c-40b1-9562-9b64c52a35b5 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:17.749880 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba7e826-eb6c-40b1-9562-9b64c52a35b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:17.780314 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-8ba7e826-eb6c-40b1-9562-9b64c52a35b5 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:00:17.781569 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-8ba7e826-eb6c-40b1-9562-9b64c52a35b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:17.877137 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 360/723] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:00:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 12:00:18.060191 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae034aa-74be-48e6-9fb7-cf01f201d052 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:18.164164 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:18.192323 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae034aa-74be-48e6-9fb7-cf01f201d052 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:18.238353 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6aef2c-ac00-45eb-b3d2-9f514aed20e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:18.239221 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6aef2c-ac00-45eb-b3d2-9f514aed20e2 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:18.239499 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6aef2c-ac00-45eb-b3d2-9f514aed20e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:18.239762 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6aef2c-ac00-45eb-b3d2-9f514aed20e2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:18.253946 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6aef2c-ac00-45eb-b3d2-9f514aed20e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:00:18.259268 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-7a6aef2c-ac00-45eb-b3d2-9f514aed20e2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 12:00:18.259268 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:18.259268 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:18.259268 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:18.259268 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:18.259268 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:18.259268 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:18.259268 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:18.259268 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:18.259268 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:18.259268 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:18.259268 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:18.259268 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:00:18.259268 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:00:18.259268 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:00:18.259268 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:00:18.260220 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:00:18.260220 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:00:18.260220 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:00:18.260220 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:00:18.260220 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:18.260220 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:18.260220 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:18.260220 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:18.260220 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:18.260220 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:18.260220 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:18.260220 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:18.260220 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:18.260220 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:18.260220 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:18.260220 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:18.261105 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:18.261105 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:18.261105 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:18.261105 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:18.261105 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:00:18.261105 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:00:18.261105 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:00:18.261105 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:00:18.261105 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:00:18.261105 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:00:18.261105 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 12:00:18.261105 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:00:18.261105 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 364/724] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Sep 9 12:00:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:18.266451 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba7e826-eb6c-40b1-9562-9b64c52a35b5 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:18.279327 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6aef2c-ac00-45eb-b3d2-9f514aed20e2 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:18.304600 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:18.424363 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba7e826-eb6c-40b1-9562-9b64c52a35b5 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:18.462325 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f78ac1-f8c5-4553-b7b4-7564c72adb96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:18.463506 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f78ac1-f8c5-4553-b7b4-7564c72adb96 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:18.463992 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f78ac1-f8c5-4553-b7b4-7564c72adb96 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:18.464788 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f78ac1-f8c5-4553-b7b4-7564c72adb96 None admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:18.465762 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f78ac1-f8c5-4553-b7b4-7564c72adb96 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:18.466299 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f78ac1-f8c5-4553-b7b4-7564c72adb96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:18.468371 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f78ac1-f8c5-4553-b7b4-7564c72adb96 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:18.479343 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 361/725] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 9 12:00:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:18.484546 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f78ac1-f8c5-4553-b7b4-7564c72adb96 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:18.489213 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6aef2c-ac00-45eb-b3d2-9f514aed20e2 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:18.544658 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be143909-0fdf-4720-9cc4-d87691a3ae5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:18.545802 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be143909-0fdf-4720-9cc4-d87691a3ae5e None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:18.546282 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be143909-0fdf-4720-9cc4-d87691a3ae5e None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:18.546749 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be143909-0fdf-4720-9cc4-d87691a3ae5e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:18.549803 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:18.559453 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be143909-0fdf-4720-9cc4-d87691a3ae5e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:00:18.565014 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-be143909-0fdf-4720-9cc4-d87691a3ae5e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 12:00:18.565014 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:18.565014 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:18.565014 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:18.565014 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:18.565014 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:18.565014 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:18.565014 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:18.565014 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:18.565014 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:18.565014 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:18.565014 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:18.565014 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:00:18.565014 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:00:18.565014 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:00:18.565014 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:00:18.566540 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:00:18.566540 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:00:18.566540 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:00:18.566540 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:00:18.566540 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:18.566540 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:18.566540 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:18.566540 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:18.566540 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:18.566540 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:18.566540 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:18.566540 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:18.566540 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:18.566540 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:18.566540 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:18.566540 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:18.567573 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:18.567573 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:18.567573 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:18.567573 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:18.567573 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:00:18.567573 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:00:18.567573 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:00:18.567573 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:00:18.567573 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:00:18.567573 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:00:18.567573 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 12:00:18.567573 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:00:18.570168 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 365/726] 10.222.0.49 () {56 vars in 1102 bytes} [Fri Sep 9 12:00:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:18.575500 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be143909-0fdf-4720-9cc4-d87691a3ae5e None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:18.666983 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:18.689035 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be143909-0fdf-4720-9cc4-d87691a3ae5e None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:18.701243 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f78ac1-f8c5-4553-b7b4-7564c72adb96 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:18.728804 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fbe2c7-940f-4fd6-bab8-79e5994d69d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:18.730006 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fbe2c7-940f-4fd6-bab8-79e5994d69d1 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:18.731282 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fbe2c7-940f-4fd6-bab8-79e5994d69d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:18.731956 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fbe2c7-940f-4fd6-bab8-79e5994d69d1 None admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:18.734037 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fbe2c7-940f-4fd6-bab8-79e5994d69d1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:18.734746 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fbe2c7-940f-4fd6-bab8-79e5994d69d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:18.737033 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fbe2c7-940f-4fd6-bab8-79e5994d69d1 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:18.756545 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 366/727] 10.222.0.49 () {56 vars in 1103 bytes} [Fri Sep 9 12:00:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:18.760637 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fbe2c7-940f-4fd6-bab8-79e5994d69d1 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:18.775710 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00bfc72-65a3-47c7-960a-13cf5de2737b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:18.776735 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00bfc72-65a3-47c7-960a-13cf5de2737b None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:18.777213 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00bfc72-65a3-47c7-960a-13cf5de2737b None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:18.777679 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00bfc72-65a3-47c7-960a-13cf5de2737b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:18.795335 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00bfc72-65a3-47c7-960a-13cf5de2737b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:18.796801 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00bfc72-65a3-47c7-960a-13cf5de2737b None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:18.831285 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:18.832751 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-d00bfc72-65a3-47c7-960a-13cf5de2737b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:00:18.832751 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:18.832751 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:18.832751 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:18.832751 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:18.832751 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:18.832751 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:18.832751 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:18.832751 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:18.832751 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:18.832751 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:18.832751 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:18.832751 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:00:18.832751 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:00:18.832751 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:00:18.832751 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:00:18.833694 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:18.833694 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:18.833694 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:18.833694 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:18.833694 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:18.833694 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:18.833694 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:18.833694 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:18.833694 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:18.833694 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:18.833694 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:18.833694 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:18.833694 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:18.833694 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:18.833694 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:18.833694 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:18.833694 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:00:18.834568 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:00:18.834568 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:00:18.834568 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:00:18.834568 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 362/728] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 12:00:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:18.848822 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00bfc72-65a3-47c7-960a-13cf5de2737b None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:18.880252 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fbe2c7-940f-4fd6-bab8-79e5994d69d1 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:18.916968 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffffc00a-2876-4aef-bf5b-4ca4e07d30a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:18.917985 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffffc00a-2876-4aef-bf5b-4ca4e07d30a4 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:18.918465 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffffc00a-2876-4aef-bf5b-4ca4e07d30a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:18.918993 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffffc00a-2876-4aef-bf5b-4ca4e07d30a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:18.931061 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffffc00a-2876-4aef-bf5b-4ca4e07d30a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:18.932756 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffffc00a-2876-4aef-bf5b-4ca4e07d30a4 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:18.948489 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-ffffc00a-2876-4aef-bf5b-4ca4e07d30a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:00:18.948489 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:18.948489 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:18.948489 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:18.948489 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:18.948489 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:18.948489 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:18.948489 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:18.948489 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:18.948489 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:18.948489 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:18.948489 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:18.948489 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:00:18.948489 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:00:18.948489 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:00:18.948489 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:00:18.949429 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:18.949429 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:18.949429 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:18.949429 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:18.949429 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:18.949429 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:18.949429 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:18.949429 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:18.949429 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:18.949429 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:18.949429 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:18.949429 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:18.949429 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:18.949429 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:18.949429 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:18.949429 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:18.949429 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:00:18.950325 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:00:18.950325 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:00:18.950325 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:00:18.952446 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 367/729] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 12:00:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:18.957773 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffffc00a-2876-4aef-bf5b-4ca4e07d30a4 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:19.067533 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffffc00a-2876-4aef-bf5b-4ca4e07d30a4 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:19.079981 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:19.101050 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00bfc72-65a3-47c7-960a-13cf5de2737b None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:19.112073 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e1f9ca-c8b3-4bfd-87bf-c000ba7c3e51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:19.112710 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e1f9ca-c8b3-4bfd-87bf-c000ba7c3e51 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:19.113512 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e1f9ca-c8b3-4bfd-87bf-c000ba7c3e51 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:19.113789 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e1f9ca-c8b3-4bfd-87bf-c000ba7c3e51 None admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:19.114436 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e1f9ca-c8b3-4bfd-87bf-c000ba7c3e51 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:19.114733 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e1f9ca-c8b3-4bfd-87bf-c000ba7c3e51 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:19.116118 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e1f9ca-c8b3-4bfd-87bf-c000ba7c3e51 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:19.142681 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 368/730] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 9 12:00:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:00:19.148913 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e1f9ca-c8b3-4bfd-87bf-c000ba7c3e51 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:19.184167 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00160ae1-a69f-44a5-add2-dc4ab89942d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:19.184788 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00160ae1-a69f-44a5-add2-dc4ab89942d9 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:19.185061 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00160ae1-a69f-44a5-add2-dc4ab89942d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:19.185326 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00160ae1-a69f-44a5-add2-dc4ab89942d9 None admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:19.185967 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00160ae1-a69f-44a5-add2-dc4ab89942d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:19.186265 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00160ae1-a69f-44a5-add2-dc4ab89942d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:19.187656 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00160ae1-a69f-44a5-add2-dc4ab89942d9 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:19.189445 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:19.221232 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 363/731] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 9 12:00:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 373 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:19.231922 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00160ae1-a69f-44a5-add2-dc4ab89942d9 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:19.268991 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e1f9ca-c8b3-4bfd-87bf-c000ba7c3e51 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:19.309624 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29361ac5-5a6a-4330-a0f4-f7f9a9d16ede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:19.311042 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29361ac5-5a6a-4330-a0f4-f7f9a9d16ede None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:19.311468 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29361ac5-5a6a-4330-a0f4-f7f9a9d16ede None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:19.311910 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29361ac5-5a6a-4330-a0f4-f7f9a9d16ede None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:19.321191 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29361ac5-5a6a-4330-a0f4-f7f9a9d16ede None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:19.323044 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29361ac5-5a6a-4330-a0f4-f7f9a9d16ede None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:19.339812 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-29361ac5-5a6a-4330-a0f4-f7f9a9d16ede None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 12:00:19.339812 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:19.339812 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:19.339812 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:19.339812 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:19.339812 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:19.339812 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:19.339812 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:19.339812 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:19.339812 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:19.339812 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:19.339812 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:19.339812 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 12:00:19.339812 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 12:00:19.339812 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 12:00:19.339812 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 12:00:19.340860 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:19.340860 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:19.340860 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 12:00:19.340860 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:00:19.340860 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 12:00:19.340860 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:00:19.340860 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:19.340860 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:19.340860 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:19.340860 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:19.340860 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:19.340860 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:19.340860 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:19.340860 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:19.340860 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:19.340860 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:19.340860 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:19.340860 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:19.341790 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:19.341790 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:19.341790 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 12:00:19.341790 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 12:00:19.341790 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 12:00:19.341790 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 12:00:19.341790 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 12:00:19.341790 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 12:00:19.341790 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 12:00:19.341790 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:00:19.341790 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 369/732] 10.222.0.49 () {56 vars in 1117 bytes} [Fri Sep 9 12:00:19 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 12:00:19.351024 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29361ac5-5a6a-4330-a0f4-f7f9a9d16ede None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:19.385208 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:19.468743 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29361ac5-5a6a-4330-a0f4-f7f9a9d16ede None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:19.494071 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00160ae1-a69f-44a5-add2-dc4ab89942d9 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:19.512173 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e046f4-c084-4044-b987-cc8fc6ef151f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:19.513166 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e046f4-c084-4044-b987-cc8fc6ef151f None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:19.513446 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e046f4-c084-4044-b987-cc8fc6ef151f None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:19.513720 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e046f4-c084-4044-b987-cc8fc6ef151f None admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:19.514451 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e046f4-c084-4044-b987-cc8fc6ef151f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:19.514816 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e046f4-c084-4044-b987-cc8fc6ef151f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:19.516837 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e046f4-c084-4044-b987-cc8fc6ef151f None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:19.530640 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d916130c-532d-4ab6-ab7a-0b14f78df9b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:19.531441 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d916130c-532d-4ab6-ab7a-0b14f78df9b6 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:19.531815 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d916130c-532d-4ab6-ab7a-0b14f78df9b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:19.532181 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d916130c-532d-4ab6-ab7a-0b14f78df9b6 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:19.538718 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 370/733] 10.222.0.49 () {56 vars in 1149 bytes} [Fri Sep 9 12:00:19 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:19.540143 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a0792e-f0de-4b3f-9f8e-8d53c05200ec None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:19.540429 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a0792e-f0de-4b3f-9f8e-8d53c05200ec None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:19.540703 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a0792e-f0de-4b3f-9f8e-8d53c05200ec None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:19.542165 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 371/734] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:00:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:00:19.545185 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a0792e-f0de-4b3f-9f8e-8d53c05200ec None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:19.547743 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d916130c-532d-4ab6-ab7a-0b14f78df9b6 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:19.549200 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d916130c-532d-4ab6-ab7a-0b14f78df9b6 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:19.567737 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-d916130c-532d-4ab6-ab7a-0b14f78df9b6 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 09 12:00:19.567737 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:19.567737 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:19.567737 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:19.567737 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:19.567737 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:19.567737 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:19.567737 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:19.567737 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:19.567737 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:19.567737 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:19.567737 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:19.567737 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 12:00:19.567737 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 12:00:19.567737 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 12:00:19.567737 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 12:00:19.568743 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:19.568743 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:19.568743 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 12:00:19.568743 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:00:19.568743 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 12:00:19.568743 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:00:19.568743 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:19.568743 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:19.568743 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:19.568743 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:19.568743 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:19.568743 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:19.568743 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:19.568743 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:19.568743 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:19.568743 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:19.568743 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:19.568743 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:19.570314 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:19.570314 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:19.570314 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 12:00:19.570314 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 12:00:19.570314 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 12:00:19.570314 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 12:00:19.570314 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 12:00:19.570314 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 12:00:19.570314 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 09 12:00:19.570314 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:00:19.574086 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 364/735] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Sep 9 12:00:19 2022] GET /identity/v3/users/swift => generated 83 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:19.575656 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8815ba5d-8343-4734-8854-0e2031043ad5 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:19.575934 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8815ba5d-8343-4734-8854-0e2031043ad5 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:19.576205 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8815ba5d-8343-4734-8854-0e2031043ad5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:19.597614 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-8815ba5d-8343-4734-8854-0e2031043ad5 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:00:19.599420 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-8815ba5d-8343-4734-8854-0e2031043ad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:19.662184 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:19.664731 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:19.686930 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 365/736] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:00:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 12:00:19.686930 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8815ba5d-8343-4734-8854-0e2031043ad5 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:19.687919 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a0792e-f0de-4b3f-9f8e-8d53c05200ec None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:19.742399 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a447909f-d562-4359-99cf-cf9fa8395dbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:19.743304 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a447909f-d562-4359-99cf-cf9fa8395dbc None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:19.743724 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a447909f-d562-4359-99cf-cf9fa8395dbc None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:19.744191 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a447909f-d562-4359-99cf-cf9fa8395dbc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:19.820670 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a447909f-d562-4359-99cf-cf9fa8395dbc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:19.822373 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a447909f-d562-4359-99cf-cf9fa8395dbc None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:19.888037 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8815ba5d-8343-4734-8854-0e2031043ad5 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:19.934361 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:19.934886 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 372/737] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 9 12:00:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 09 12:00:19.940686 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a447909f-d562-4359-99cf-cf9fa8395dbc None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:19.979442 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e57ce0-c7f6-4f17-bf85-21ec7c9f8767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:19.979442 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e57ce0-c7f6-4f17-bf85-21ec7c9f8767 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:19.979722 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e57ce0-c7f6-4f17-bf85-21ec7c9f8767 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:19.979955 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e57ce0-c7f6-4f17-bf85-21ec7c9f8767 None admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:19.980694 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e57ce0-c7f6-4f17-bf85-21ec7c9f8767 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:19.980995 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e57ce0-c7f6-4f17-bf85-21ec7c9f8767 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:19.982434 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e57ce0-c7f6-4f17-bf85-21ec7c9f8767 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:20.029329 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 366/738] 10.222.0.49 () {56 vars in 1135 bytes} [Fri Sep 9 12:00:19 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:20.049070 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e57ce0-c7f6-4f17-bf85-21ec7c9f8767 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:20.065166 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a447909f-d562-4359-99cf-cf9fa8395dbc None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:20.106447 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0778424b-5573-40f1-b2b2-a740c8cb5102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:20.107501 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0778424b-5573-40f1-b2b2-a740c8cb5102 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:20.107979 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0778424b-5573-40f1-b2b2-a740c8cb5102 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:20.108458 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0778424b-5573-40f1-b2b2-a740c8cb5102 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:20.122359 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0778424b-5573-40f1-b2b2-a740c8cb5102 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:20.124169 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0778424b-5573-40f1-b2b2-a740c8cb5102 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:20.130418 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:20.147767 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-0778424b-5573-40f1-b2b2-a740c8cb5102 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:00:20.147767 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:20.147767 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:20.147767 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:20.147767 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:20.147767 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:20.147767 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:20.147767 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:20.147767 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:20.147767 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:20.147767 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:20.147767 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:20.147767 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:00:20.147767 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:00:20.147767 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:00:20.147767 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:00:20.148779 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:20.148779 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:20.148779 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:20.148779 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:20.148779 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:20.148779 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:20.148779 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:20.148779 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:20.148779 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:20.148779 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:20.148779 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:20.148779 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:20.148779 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:20.148779 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:20.148779 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:20.148779 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:20.148779 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:00:20.149660 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:00:20.149660 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:00:20.149660 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:00:20.152137 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 373/739] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 12:00:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:20.153292 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0778424b-5573-40f1-b2b2-a740c8cb5102 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:20.197785 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e57ce0-c7f6-4f17-bf85-21ec7c9f8767 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:20.200878 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:20.245284 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a76235a-485f-4f4d-8cf2-46f5ed6682a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:20.245909 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a76235a-485f-4f4d-8cf2-46f5ed6682a9 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:20.246197 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a76235a-485f-4f4d-8cf2-46f5ed6682a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:20.246469 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a76235a-485f-4f4d-8cf2-46f5ed6682a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:20.270151 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0778424b-5573-40f1-b2b2-a740c8cb5102 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:20.308761 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c7c83f-c419-4234-8789-b08f9a60c7a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:20.309659 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c7c83f-c419-4234-8789-b08f9a60c7a2 None admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:20.310076 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c7c83f-c419-4234-8789-b08f9a60c7a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:20.310482 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c7c83f-c419-4234-8789-b08f9a60c7a2 None admin] PATH_INFO: `/v3/services` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:20.311288 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c7c83f-c419-4234-8789-b08f9a60c7a2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:20.312875 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c7c83f-c419-4234-8789-b08f9a60c7a2 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:20.315606 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a76235a-485f-4f4d-8cf2-46f5ed6682a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:20.317082 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a76235a-485f-4f4d-8cf2-46f5ed6682a9 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:20.333246 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 374/740] 10.222.0.49 () {60 vars in 1141 bytes} [Fri Sep 9 12:00:20 2022] POST /identity/v3/services => generated 251 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:00:20.334741 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af555b55-aa24-4c9d-8abb-246288424b34 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:20.335193 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af555b55-aa24-4c9d-8abb-246288424b34 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:20.335681 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af555b55-aa24-4c9d-8abb-246288424b34 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:20.337083 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 375/741] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:00:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:00:20.338014 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af555b55-aa24-4c9d-8abb-246288424b34 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:20.378211 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:20.442529 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 367/742] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 9 12:00:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 12:00:20.443935 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61533187-7655-4dcf-b69c-07571bce3d1d None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:20.444212 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61533187-7655-4dcf-b69c-07571bce3d1d None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:20.444482 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61533187-7655-4dcf-b69c-07571bce3d1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:20.463347 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-61533187-7655-4dcf-b69c-07571bce3d1d None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:00:20.464577 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-61533187-7655-4dcf-b69c-07571bce3d1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:20.468469 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:20.488274 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af555b55-aa24-4c9d-8abb-246288424b34 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:20.531657 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0f0509-e897-4818-9287-2b707faca23e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:20.532275 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0f0509-e897-4818-9287-2b707faca23e None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:20.532562 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0f0509-e897-4818-9287-2b707faca23e None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:20.533525 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0f0509-e897-4818-9287-2b707faca23e None admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:20.533525 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0f0509-e897-4818-9287-2b707faca23e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:20.533727 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0f0509-e897-4818-9287-2b707faca23e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:20.535957 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0f0509-e897-4818-9287-2b707faca23e None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:20.543686 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 368/743] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:00:20 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 12:00:20.550922 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac3e954-492b-4f18-a328-e90926f72d54 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:20.550922 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac3e954-492b-4f18-a328-e90926f72d54 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:20.550922 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac3e954-492b-4f18-a328-e90926f72d54 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:20.551298 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 376/744] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 9 12:00:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:20.551401 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 369/745] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:00:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:00:20.551486 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0f0509-e897-4818-9287-2b707faca23e None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:20.556816 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac3e954-492b-4f18-a328-e90926f72d54 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:20.649010 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:20.680673 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0f0509-e897-4818-9287-2b707faca23e None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:20.694598 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac3e954-492b-4f18-a328-e90926f72d54 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:20.718742 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b57ba0-2bb7-4c98-a840-8a2d4e5413b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:20.719731 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b57ba0-2bb7-4c98-a840-8a2d4e5413b8 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:20.720178 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b57ba0-2bb7-4c98-a840-8a2d4e5413b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:20.720604 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b57ba0-2bb7-4c98-a840-8a2d4e5413b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:20.732501 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25c610f-620c-4b18-a4ba-a7be2cbdd818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:20.733139 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25c610f-620c-4b18-a4ba-a7be2cbdd818 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:20.733327 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b57ba0-2bb7-4c98-a840-8a2d4e5413b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:20.733437 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25c610f-620c-4b18-a4ba-a7be2cbdd818 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:20.733669 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25c610f-620c-4b18-a4ba-a7be2cbdd818 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:20.735577 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b57ba0-2bb7-4c98-a840-8a2d4e5413b8 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:20.742112 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25c610f-620c-4b18-a4ba-a7be2cbdd818 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:20.743868 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25c610f-620c-4b18-a4ba-a7be2cbdd818 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:20.752045 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-16b57ba0-2bb7-4c98-a840-8a2d4e5413b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:00:20.752045 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:20.752045 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:20.752045 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:20.752045 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:20.752045 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:20.752045 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:20.752045 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:20.752045 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:20.752045 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:20.752045 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:20.752045 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:20.752045 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:00:20.752045 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:00:20.752045 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:00:20.752045 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:00:20.753273 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:20.753273 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:20.753273 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:20.753273 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:20.753273 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:20.753273 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:20.753273 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:20.753273 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:20.753273 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:20.753273 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:20.753273 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:20.753273 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:20.753273 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:20.753273 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:20.753273 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:20.753273 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:20.753273 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:00:20.755066 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:00:20.755066 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:00:20.755066 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:00:20.756801 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 377/746] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 12:00:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:20.758155 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d92893-c4bc-40f5-ba27-5e88698f007c None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:20.758593 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d92893-c4bc-40f5-ba27-5e88698f007c None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:20.759182 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d92893-c4bc-40f5-ba27-5e88698f007c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:20.759809 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-a25c610f-620c-4b18-a4ba-a7be2cbdd818 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:00:20.759809 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:20.759809 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:20.759809 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:20.759809 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:20.759809 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:20.759809 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:20.759809 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:20.759809 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:20.759809 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:20.759809 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:20.759809 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:20.759809 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 12:00:20.759809 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 12:00:20.759809 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 12:00:20.760777 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 12:00:20.760777 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:20.760777 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:20.760777 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:20.760777 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:20.760777 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:20.760777 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:20.760777 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:20.760777 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:20.760777 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:20.760777 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:20.760777 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:20.760777 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:20.760777 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:20.760777 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:20.760777 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:20.760777 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:20.761682 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 12:00:20.761682 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 12:00:20.761682 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 12:00:20.761682 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 12:00:20.761682 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 12:00:20.761682 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 12:00:20.761682 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:00:20.761682 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:00:20.761682 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 370/747] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 9 12:00:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:20.767182 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25c610f-620c-4b18-a4ba-a7be2cbdd818 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:20.788485 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-a5d92893-c4bc-40f5-ba27-5e88698f007c None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:00:20.789931 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-a5d92893-c4bc-40f5-ba27-5e88698f007c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:20.792234 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:20.865355 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 378/748] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:00:20 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 12:00:20.866452 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d92893-c4bc-40f5-ba27-5e88698f007c None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:20.884909 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25c610f-620c-4b18-a4ba-a7be2cbdd818 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:20.925487 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:20.933518 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9664b4-6221-41f7-8dff-9efda320c3b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:20.935037 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9664b4-6221-41f7-8dff-9efda320c3b7 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:20.935501 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9664b4-6221-41f7-8dff-9efda320c3b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:20.936189 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9664b4-6221-41f7-8dff-9efda320c3b7 None admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:20.937280 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9664b4-6221-41f7-8dff-9efda320c3b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:20.937763 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9664b4-6221-41f7-8dff-9efda320c3b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:20.940295 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9664b4-6221-41f7-8dff-9efda320c3b7 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:20.958285 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 371/749] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 9 12:00:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:20.959925 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9664b4-6221-41f7-8dff-9efda320c3b7 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:20.982395 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d92893-c4bc-40f5-ba27-5e88698f007c None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:21.021128 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aee18b-09fe-4e0d-8471-f9ac8591b11a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:21.021908 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aee18b-09fe-4e0d-8471-f9ac8591b11a None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:21.022322 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aee18b-09fe-4e0d-8471-f9ac8591b11a None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:21.022662 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aee18b-09fe-4e0d-8471-f9ac8591b11a None admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:21.023738 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31aee18b-09fe-4e0d-8471-f9ac8591b11a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:21.023981 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31aee18b-09fe-4e0d-8471-f9ac8591b11a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:21.025659 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31aee18b-09fe-4e0d-8471-f9ac8591b11a None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:21.032993 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:21.040469 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 379/750] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Sep 9 12:00:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:21.046256 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aee18b-09fe-4e0d-8471-f9ac8591b11a None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:21.090168 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9664b4-6221-41f7-8dff-9efda320c3b7 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:21.134589 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30adbdb-8b31-457f-a6ef-c3432110adc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:21.135841 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30adbdb-8b31-457f-a6ef-c3432110adc9 None admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:21.136305 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30adbdb-8b31-457f-a6ef-c3432110adc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:21.136772 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30adbdb-8b31-457f-a6ef-c3432110adc9 None admin] PATH_INFO: `/v3/services` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:21.137726 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30adbdb-8b31-457f-a6ef-c3432110adc9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:21.139913 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30adbdb-8b31-457f-a6ef-c3432110adc9 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:21.158936 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aee18b-09fe-4e0d-8471-f9ac8591b11a None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:21.162951 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 372/751] 10.222.0.49 () {60 vars in 1141 bytes} [Fri Sep 9 12:00:20 2022] POST /identity/v3/services => generated 240 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:00:21.162951 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30adbdb-8b31-457f-a6ef-c3432110adc9 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:21.215967 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3db1865-dd0c-4b87-a58d-f54ef7ac9a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:21.217394 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3db1865-dd0c-4b87-a58d-f54ef7ac9a94 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:21.217850 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3db1865-dd0c-4b87-a58d-f54ef7ac9a94 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:21.218269 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3db1865-dd0c-4b87-a58d-f54ef7ac9a94 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:21.228014 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3db1865-dd0c-4b87-a58d-f54ef7ac9a94 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:21.229487 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3db1865-dd0c-4b87-a58d-f54ef7ac9a94 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:21.235414 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:21.244138 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-c3db1865-dd0c-4b87-a58d-f54ef7ac9a94 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:00:21.244138 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:21.244138 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:21.244138 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:21.244138 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:21.244138 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:21.244138 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:21.244138 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:21.244138 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:21.244138 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:21.244138 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:21.244138 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:21.244138 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 12:00:21.244138 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 12:00:21.244138 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 12:00:21.245318 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 12:00:21.245318 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:21.245318 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:21.245318 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:21.245318 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:21.245318 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:21.245318 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:21.245318 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:21.245318 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:21.245318 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:21.245318 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:21.245318 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:21.245318 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:21.245318 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:21.245318 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:21.245318 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:21.245318 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:21.246397 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 12:00:21.246397 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 12:00:21.246397 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 12:00:21.246397 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 12:00:21.246397 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 12:00:21.246397 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 12:00:21.246397 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:00:21.246397 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:00:21.246397 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 380/752] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 9 12:00:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:21.246923 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3db1865-dd0c-4b87-a58d-f54ef7ac9a94 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:21.324051 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30adbdb-8b31-457f-a6ef-c3432110adc9 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:21.329006 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:21.362884 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b91bdce-fb11-4a26-8cd4-fbeb048229f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:21.363598 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b91bdce-fb11-4a26-8cd4-fbeb048229f7 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:21.363877 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b91bdce-fb11-4a26-8cd4-fbeb048229f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:21.364148 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b91bdce-fb11-4a26-8cd4-fbeb048229f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:21.364873 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b91bdce-fb11-4a26-8cd4-fbeb048229f7 None admin] RBAC: Adding query filter params (role.id=8abeff2cb87d4b7fa72b27b028005877, scope.project.id=fe863fcd75f1406ebbffc5016078f73e, user.id=e368b4e668c54e25bff80a8e8a8c3588) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:21.365194 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b91bdce-fb11-4a26-8cd4-fbeb048229f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:21.366612 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b91bdce-fb11-4a26-8cd4-fbeb048229f7 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:21.373417 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 373/753] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Sep 9 12:00:21 2022] GET /identity/v3/role_assignments?user.id=e368b4e668c54e25bff80a8e8a8c3588&scope.project.id=fe863fcd75f1406ebbffc5016078f73e&role.id=8abeff2cb87d4b7fa72b27b028005877 => generated 259 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:21.374270 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b91bdce-fb11-4a26-8cd4-fbeb048229f7 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:21.398561 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3db1865-dd0c-4b87-a58d-f54ef7ac9a94 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:21.438325 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bc3f91-c66d-4076-8465-42200af42a9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:21.439500 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bc3f91-c66d-4076-8465-42200af42a9f None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:21.440246 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bc3f91-c66d-4076-8465-42200af42a9f None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:21.440753 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bc3f91-c66d-4076-8465-42200af42a9f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:21.453915 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bc3f91-c66d-4076-8465-42200af42a9f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:00:21.460642 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-b7bc3f91-c66d-4076-8465-42200af42a9f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 12:00:21.460642 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:21.460642 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:21.460642 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:21.460642 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:21.460642 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:21.460642 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:21.460642 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:21.460642 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:21.460642 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:21.460642 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:21.460642 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:21.460642 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:00:21.460642 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:00:21.460642 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:00:21.460642 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:00:21.461633 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:00:21.461633 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:00:21.461633 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:00:21.461633 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:00:21.461633 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:21.461633 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:21.461633 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:21.461633 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:21.461633 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:21.461633 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:21.461633 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:21.461633 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:21.461633 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:21.461633 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:21.461633 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:21.461633 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:21.462482 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:21.462482 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:21.462482 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:21.462482 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:21.462482 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:00:21.462482 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:00:21.462482 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:00:21.462482 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:00:21.462482 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:00:21.462482 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:00:21.462482 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 12:00:21.462482 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:00:21.465351 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 381/754] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Sep 9 12:00:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:21.472660 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bc3f91-c66d-4076-8465-42200af42a9f None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:21.497723 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b91bdce-fb11-4a26-8cd4-fbeb048229f7 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:21.516345 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:21.536822 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12816ee3-5700-4042-8983-512e7d724a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:21.537417 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12816ee3-5700-4042-8983-512e7d724a66 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:21.537697 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12816ee3-5700-4042-8983-512e7d724a66 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:21.537967 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12816ee3-5700-4042-8983-512e7d724a66 None admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:21.538955 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12816ee3-5700-4042-8983-512e7d724a66 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:21.538955 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12816ee3-5700-4042-8983-512e7d724a66 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:21.540500 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12816ee3-5700-4042-8983-512e7d724a66 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:21.555429 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 374/755] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Sep 9 12:00:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:21.568034 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12816ee3-5700-4042-8983-512e7d724a66 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:21.588950 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bc3f91-c66d-4076-8465-42200af42a9f None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:21.616632 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:21.641661 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767ce102-596f-42af-8141-fefc638d1b67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:21.642283 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767ce102-596f-42af-8141-fefc638d1b67 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:21.642564 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767ce102-596f-42af-8141-fefc638d1b67 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:21.642831 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767ce102-596f-42af-8141-fefc638d1b67 None admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:21.643528 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767ce102-596f-42af-8141-fefc638d1b67 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:21.643854 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767ce102-596f-42af-8141-fefc638d1b67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:21.645903 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767ce102-596f-42af-8141-fefc638d1b67 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:21.655508 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 382/756] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 9 12:00:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:00:21.667966 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767ce102-596f-42af-8141-fefc638d1b67 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:21.684003 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12816ee3-5700-4042-8983-512e7d724a66 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:21.719963 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13bd0f1-fd47-4947-b990-ea92258efb0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:21.720582 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13bd0f1-fd47-4947-b990-ea92258efb0b None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:21.720848 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13bd0f1-fd47-4947-b990-ea92258efb0b None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:21.721124 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13bd0f1-fd47-4947-b990-ea92258efb0b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:21.721848 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13bd0f1-fd47-4947-b990-ea92258efb0b None admin] RBAC: Adding query filter params (role.id=3f2ca33902a04b6dbc2ef7d06d7289d3, scope.project.id=fe863fcd75f1406ebbffc5016078f73e, user.id=92e652ba827849b5a3fc1213d04a1b76) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:21.722215 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13bd0f1-fd47-4947-b990-ea92258efb0b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:21.724060 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13bd0f1-fd47-4947-b990-ea92258efb0b None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:21.730186 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 375/757] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Sep 9 12:00:21 2022] GET /identity/v3/role_assignments?user.id=92e652ba827849b5a3fc1213d04a1b76&scope.project.id=fe863fcd75f1406ebbffc5016078f73e&role.id=3f2ca33902a04b6dbc2ef7d06d7289d3 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:21.812384 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767ce102-596f-42af-8141-fefc638d1b67 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:21.849268 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a01e6f8-bd68-4c96-a63c-d6693ae5e012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:21.849907 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a01e6f8-bd68-4c96-a63c-d6693ae5e012 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:21.850185 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a01e6f8-bd68-4c96-a63c-d6693ae5e012 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:21.850453 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a01e6f8-bd68-4c96-a63c-d6693ae5e012 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:21.862379 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a01e6f8-bd68-4c96-a63c-d6693ae5e012 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:21.864238 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a01e6f8-bd68-4c96-a63c-d6693ae5e012 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:21.880144 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-5a01e6f8-bd68-4c96-a63c-d6693ae5e012 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:00:21.880144 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:21.880144 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:21.880144 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:21.880144 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:21.880144 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:21.880144 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:21.880144 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:21.880144 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:21.880144 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:21.880144 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:21.880144 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:21.880144 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:00:21.880144 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:00:21.880144 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:00:21.880144 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:00:21.881185 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:21.881185 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:21.881185 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:21.881185 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:21.881185 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:21.881185 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:21.881185 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:21.881185 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:21.881185 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:21.881185 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:21.881185 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:21.881185 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:21.881185 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:21.881185 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:21.881185 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:21.881185 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:21.881185 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:00:21.882078 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:00:21.882078 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:00:21.882078 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:00:21.886115 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 383/758] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 12:00:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 12:00:21.887609 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13bd0f1-fd47-4947-b990-ea92258efb0b None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:21.935296 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:21.998456 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13bd0f1-fd47-4947-b990-ea92258efb0b None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:22.035648 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64a5639-edb8-4362-8ac9-c35fa21ccb19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:22.036260 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64a5639-edb8-4362-8ac9-c35fa21ccb19 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:22.036536 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64a5639-edb8-4362-8ac9-c35fa21ccb19 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:22.036805 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64a5639-edb8-4362-8ac9-c35fa21ccb19 None admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:22.037445 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64a5639-edb8-4362-8ac9-c35fa21ccb19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:22.037747 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64a5639-edb8-4362-8ac9-c35fa21ccb19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:22.039144 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64a5639-edb8-4362-8ac9-c35fa21ccb19 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:22.057307 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 376/759] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 9 12:00:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:22.058646 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a01e6f8-bd68-4c96-a63c-d6693ae5e012 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:22.096202 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:22.167960 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a01e6f8-bd68-4c96-a63c-d6693ae5e012 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:22.203596 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce02f85-6e87-40e2-b9c9-d5f6a54fa1c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:22.204227 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce02f85-6e87-40e2-b9c9-d5f6a54fa1c5 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:22.204554 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce02f85-6e87-40e2-b9c9-d5f6a54fa1c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:22.204830 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce02f85-6e87-40e2-b9c9-d5f6a54fa1c5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:22.213280 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce02f85-6e87-40e2-b9c9-d5f6a54fa1c5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:22.214748 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce02f85-6e87-40e2-b9c9-d5f6a54fa1c5 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:22.226679 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-dce02f85-6e87-40e2-b9c9-d5f6a54fa1c5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 12:00:22.226679 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:22.226679 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:22.226679 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:22.226679 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:22.226679 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:22.226679 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:22.226679 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:22.226679 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:22.226679 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:22.226679 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:22.226679 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:22.226679 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 12:00:22.226679 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 12:00:22.226679 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 12:00:22.226679 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 12:00:22.227698 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:22.227698 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:22.227698 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 12:00:22.227698 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:00:22.227698 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 12:00:22.227698 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:00:22.227698 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:22.227698 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:22.227698 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:22.227698 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:22.227698 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:22.227698 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:22.227698 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:22.227698 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:22.227698 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:22.227698 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:22.227698 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:22.227698 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:22.228631 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:22.228631 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:22.228631 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 12:00:22.228631 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 12:00:22.228631 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 12:00:22.228631 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 12:00:22.228631 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 12:00:22.228631 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 12:00:22.228631 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 12:00:22.228631 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:00:22.232550 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 384/760] 10.222.0.49 () {56 vars in 1093 bytes} [Fri Sep 9 12:00:22 2022] GET /identity/v3/users/nova => generated 82 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:22.233897 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64a5639-edb8-4362-8ac9-c35fa21ccb19 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:22.338951 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:22.342018 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64a5639-edb8-4362-8ac9-c35fa21ccb19 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:22.378635 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6cb522-e3be-4f54-a8a5-354489ec79f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:22.379278 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6cb522-e3be-4f54-a8a5-354489ec79f2 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:22.379550 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6cb522-e3be-4f54-a8a5-354489ec79f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:22.379818 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6cb522-e3be-4f54-a8a5-354489ec79f2 None admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:22.380599 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6cb522-e3be-4f54-a8a5-354489ec79f2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:22.380902 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6cb522-e3be-4f54-a8a5-354489ec79f2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:22.382384 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6cb522-e3be-4f54-a8a5-354489ec79f2 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:22.405472 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 377/761] 10.222.0.49 () {56 vars in 1133 bytes} [Fri Sep 9 12:00:22 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:22.406813 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce02f85-6e87-40e2-b9c9-d5f6a54fa1c5 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:22.547075 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:22.562934 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce02f85-6e87-40e2-b9c9-d5f6a54fa1c5 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:22.613604 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2441f9-f834-45e9-a673-9c656d5bd32c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:22.614313 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2441f9-f834-45e9-a673-9c656d5bd32c None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:22.614589 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2441f9-f834-45e9-a673-9c656d5bd32c None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:22.614855 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2441f9-f834-45e9-a673-9c656d5bd32c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:22.634155 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2441f9-f834-45e9-a673-9c656d5bd32c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:22.634155 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2441f9-f834-45e9-a673-9c656d5bd32c None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:22.652252 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-4b2441f9-f834-45e9-a673-9c656d5bd32c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:00:22.652252 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:22.652252 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:22.652252 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:22.652252 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:22.652252 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:22.652252 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:22.652252 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:22.652252 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:22.652252 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:22.652252 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:22.652252 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:22.652252 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:00:22.652252 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:00:22.652252 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:00:22.652252 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:00:22.653207 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:22.653207 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:22.653207 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:22.653207 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:22.653207 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:22.653207 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:22.653207 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:22.653207 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:22.653207 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:22.653207 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:22.653207 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:22.653207 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:22.653207 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:22.653207 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:22.653207 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:22.653207 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:22.653207 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:00:22.654104 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:00:22.654104 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:00:22.654104 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:00:22.654104 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 385/762] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 12:00:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:22.659105 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6cb522-e3be-4f54-a8a5-354489ec79f2 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:22.773976 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6cb522-e3be-4f54-a8a5-354489ec79f2 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:22.814464 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da3dbc5-8708-4bd5-a144-ebce8eaced92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:22.815366 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da3dbc5-8708-4bd5-a144-ebce8eaced92 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:22.816222 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da3dbc5-8708-4bd5-a144-ebce8eaced92 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:22.816620 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da3dbc5-8708-4bd5-a144-ebce8eaced92 None admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:22.817266 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da3dbc5-8708-4bd5-a144-ebce8eaced92 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:22.817569 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da3dbc5-8708-4bd5-a144-ebce8eaced92 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:22.818942 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da3dbc5-8708-4bd5-a144-ebce8eaced92 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:22.835029 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 378/763] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 9 12:00:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:22.839978 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2441f9-f834-45e9-a673-9c656d5bd32c None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:23.018596 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2441f9-f834-45e9-a673-9c656d5bd32c None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:23.053560 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a2b12c-aa48-4cdd-934a-64062b1d3df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:23.054196 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a2b12c-aa48-4cdd-934a-64062b1d3df8 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:23.054475 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a2b12c-aa48-4cdd-934a-64062b1d3df8 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:23.054747 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a2b12c-aa48-4cdd-934a-64062b1d3df8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:23.061594 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a2b12c-aa48-4cdd-934a-64062b1d3df8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:23.063077 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a2b12c-aa48-4cdd-934a-64062b1d3df8 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:23.065893 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:23.075233 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-87a2b12c-aa48-4cdd-934a-64062b1d3df8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:00:23.075233 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:23.075233 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:23.075233 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:23.075233 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:23.075233 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:23.075233 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:23.075233 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:23.075233 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:23.075233 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:23.075233 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:23.075233 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:23.075233 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 12:00:23.075233 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 12:00:23.075233 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 12:00:23.076375 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 12:00:23.076375 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:23.076375 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:23.076375 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:23.076375 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:23.076375 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:23.076375 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:23.076375 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:23.076375 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:23.076375 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:23.076375 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:23.076375 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:23.076375 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:23.076375 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:23.076375 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:23.076375 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:23.076375 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:23.077265 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 12:00:23.077265 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 12:00:23.077265 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 12:00:23.077265 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 12:00:23.077265 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 12:00:23.077265 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 12:00:23.077265 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:00:23.077265 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:00:23.080036 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 386/764] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 9 12:00:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:23.086406 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da3dbc5-8708-4bd5-a144-ebce8eaced92 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:23.091038 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83a361e-f7c5-4c5c-b230-b0766bd9b222 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:23.091328 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83a361e-f7c5-4c5c-b230-b0766bd9b222 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:23.091605 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83a361e-f7c5-4c5c-b230-b0766bd9b222 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:23.093161 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 387/765] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:00:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 12:00:23.099876 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0b280e-0c69-4727-a2fc-cac4ae73eb2e None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:23.100154 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0b280e-0c69-4727-a2fc-cac4ae73eb2e None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:23.100429 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0b280e-0c69-4727-a2fc-cac4ae73eb2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:23.135295 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:23.135496 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-3e0b280e-0c69-4727-a2fc-cac4ae73eb2e None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:00:23.136652 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-3e0b280e-0c69-4727-a2fc-cac4ae73eb2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:23.252935 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 388/766] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:00:23 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 12:00:23.277155 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da3dbc5-8708-4bd5-a144-ebce8eaced92 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:23.333536 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9ff299-6c75-42cb-8a66-dd323d6e1adb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:23.334153 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9ff299-6c75-42cb-8a66-dd323d6e1adb None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:23.334430 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9ff299-6c75-42cb-8a66-dd323d6e1adb None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:23.334699 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9ff299-6c75-42cb-8a66-dd323d6e1adb None admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:23.335421 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9ff299-6c75-42cb-8a66-dd323d6e1adb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:23.335722 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9ff299-6c75-42cb-8a66-dd323d6e1adb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:23.337160 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9ff299-6c75-42cb-8a66-dd323d6e1adb None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:23.354185 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 379/767] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Sep 9 12:00:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:23.359832 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0b280e-0c69-4727-a2fc-cac4ae73eb2e None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:23.484412 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0b280e-0c69-4727-a2fc-cac4ae73eb2e None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:23.524800 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:23.535398 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930d0282-3910-4efa-8f61-247a079eaae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:23.536547 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930d0282-3910-4efa-8f61-247a079eaae2 None admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:23.537081 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930d0282-3910-4efa-8f61-247a079eaae2 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:23.537679 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930d0282-3910-4efa-8f61-247a079eaae2 None admin] PATH_INFO: `/v3/projects/fe863fcd75f1406ebbffc5016078f73e/users/70bac169d5cd43e1926f310344ac11fa/roles/3f2ca33902a04b6dbc2ef7d06d7289d3` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:23.540018 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930d0282-3910-4efa-8f61-247a079eaae2 None admin] RBAC: Authorizing `identity:create_grant(project_id=fe863fcd75f1406ebbffc5016078f73e, user_id=70bac169d5cd43e1926f310344ac11fa, role_id=3f2ca33902a04b6dbc2ef7d06d7289d3)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:23.541504 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930d0282-3910-4efa-8f61-247a079eaae2 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:23.567638 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 389/768] 10.222.0.49 () {58 vars in 1439 bytes} [Fri Sep 9 12:00:23 2022] PUT /identity/v3/projects/fe863fcd75f1406ebbffc5016078f73e/users/70bac169d5cd43e1926f310344ac11fa/roles/3f2ca33902a04b6dbc2ef7d06d7289d3 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:00:23.723530 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9ff299-6c75-42cb-8a66-dd323d6e1adb None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:23.826225 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:23.852737 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9ff299-6c75-42cb-8a66-dd323d6e1adb None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:23.891822 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5f11bd-d7ac-4c3b-b1e6-417e474cb7bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:23.892470 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5f11bd-d7ac-4c3b-b1e6-417e474cb7bc None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:23.892747 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5f11bd-d7ac-4c3b-b1e6-417e474cb7bc None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:23.893016 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5f11bd-d7ac-4c3b-b1e6-417e474cb7bc None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:23.893603 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5f11bd-d7ac-4c3b-b1e6-417e474cb7bc None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:23.895959 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5f11bd-d7ac-4c3b-b1e6-417e474cb7bc None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:23.906113 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-ae5f11bd-d7ac-4c3b-b1e6-417e474cb7bc None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 12:00:23.906113 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:23.906113 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 12:00:23.906113 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 12:00:23.906113 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 12:00:23.906113 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 12:00:23.906113 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 12:00:23.906113 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 12:00:23.906113 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 12:00:23.906113 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:00:23.906113 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 12:00:23.906113 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:00:23.906113 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:23.906113 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:23.906113 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:23.906113 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:23.906113 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:23.907258 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:23.907258 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:23.907258 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:23.907258 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:23.907258 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:23.907258 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:23.907258 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 12:00:23.907258 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 12:00:23.907258 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 12:00:23.907258 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 12:00:23.907258 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:23.907258 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:23.907258 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:23.907258 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:23.907258 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:23.907258 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:23.908110 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:23.908110 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:23.908110 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:23.908110 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:23.908110 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:23.908110 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:23.908110 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:23.908110 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:23.908110 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:23.908110 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:23.908110 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 12:00:23.908110 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 12:00:23.908110 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 12:00:23.908110 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:00:23.908110 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 380/769] 10.222.0.49 () {56 vars in 1129 bytes} [Fri Sep 9 12:00:23 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 12:00:23.914486 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930d0282-3910-4efa-8f61-247a079eaae2 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:24.035545 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930d0282-3910-4efa-8f61-247a079eaae2 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:24.061004 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e0d9ff-f46d-49d6-9c76-de46e2c11a07 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:24.061361 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e0d9ff-f46d-49d6-9c76-de46e2c11a07 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:24.061532 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e0d9ff-f46d-49d6-9c76-de46e2c11a07 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:24.062871 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 381/770] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:00:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 12:00:24.068953 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e6d2ed-78ab-4d09-9aba-4ce19768c47e None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:24.069239 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e6d2ed-78ab-4d09-9aba-4ce19768c47e None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:24.069509 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e6d2ed-78ab-4d09-9aba-4ce19768c47e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:24.080710 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e277b5fa-6dfe-4a9e-91e0-39f734063d10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:24.081639 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e277b5fa-6dfe-4a9e-91e0-39f734063d10 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:24.082169 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e277b5fa-6dfe-4a9e-91e0-39f734063d10 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:24.082621 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e277b5fa-6dfe-4a9e-91e0-39f734063d10 None admin] PATH_INFO: `/v3/services` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:24.083480 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e277b5fa-6dfe-4a9e-91e0-39f734063d10 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:24.083977 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e277b5fa-6dfe-4a9e-91e0-39f734063d10 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:24.085531 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e277b5fa-6dfe-4a9e-91e0-39f734063d10 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:24.096348 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 390/771] 10.222.0.49 () {56 vars in 1124 bytes} [Fri Sep 9 12:00:23 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:24.100031 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-99e6d2ed-78ab-4d09-9aba-4ce19768c47e None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:00:24.101227 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-99e6d2ed-78ab-4d09-9aba-4ce19768c47e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:24.103187 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e277b5fa-6dfe-4a9e-91e0-39f734063d10 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:24.172293 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:24.184849 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 382/772] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:00:24 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 12:00:24.186695 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3aeb0d-6d57-4f6b-8961-a6c409e4cc80 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:24.187004 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3aeb0d-6d57-4f6b-8961-a6c409e4cc80 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:24.187280 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3aeb0d-6d57-4f6b-8961-a6c409e4cc80 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:24.193275 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 383/773] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:00:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:00:24.194879 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68ff74f-71e3-4c93-9639-aefd6c2691c4 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:24.195184 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68ff74f-71e3-4c93-9639-aefd6c2691c4 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:24.195452 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68ff74f-71e3-4c93-9639-aefd6c2691c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:24.215523 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-f68ff74f-71e3-4c93-9639-aefd6c2691c4 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:00:24.216699 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-f68ff74f-71e3-4c93-9639-aefd6c2691c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:24.228428 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e277b5fa-6dfe-4a9e-91e0-39f734063d10 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:24.269793 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fb0951-5fc0-4f68-b29e-3a559d5aeb7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:24.271129 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fb0951-5fc0-4f68-b29e-3a559d5aeb7b None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:24.271739 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fb0951-5fc0-4f68-b29e-3a559d5aeb7b None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:24.272221 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fb0951-5fc0-4f68-b29e-3a559d5aeb7b None admin] PATH_INFO: `/v3/services` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:24.273060 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fb0951-5fc0-4f68-b29e-3a559d5aeb7b None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:24.273577 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fb0951-5fc0-4f68-b29e-3a559d5aeb7b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:24.275322 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fb0951-5fc0-4f68-b29e-3a559d5aeb7b None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:24.282325 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 391/774] 10.222.0.49 () {56 vars in 1124 bytes} [Fri Sep 9 12:00:24 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:24.288956 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fb0951-5fc0-4f68-b29e-3a559d5aeb7b None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:24.302989 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 384/775] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:00:24 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 12:00:24.397989 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fb0951-5fc0-4f68-b29e-3a559d5aeb7b None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:24.419313 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:24.434159 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9e8912-3c9a-4490-9e89-d61411c00836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:24.435150 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9e8912-3c9a-4490-9e89-d61411c00836 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:24.435698 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9e8912-3c9a-4490-9e89-d61411c00836 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:24.436170 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9e8912-3c9a-4490-9e89-d61411c00836 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:24.437278 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9e8912-3c9a-4490-9e89-d61411c00836 None admin] RBAC: Adding query filter params (interface=public, service_id=ae429d15f6db47d28381e5cf42243166, region_id=RegionOne) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:24.437851 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9e8912-3c9a-4490-9e89-d61411c00836 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:24.439513 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9e8912-3c9a-4490-9e89-d61411c00836 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:24.448775 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 392/776] 10.222.0.49 () {56 vars in 1251 bytes} [Fri Sep 9 12:00:24 2022] GET /identity/v3/endpoints?service_id=ae429d15f6db47d28381e5cf42243166&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:24.454534 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68ff74f-71e3-4c93-9639-aefd6c2691c4 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:24.457670 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:24.488041 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f552e063-422f-440f-995d-ee82d62d7dc5 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:24.488621 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f552e063-422f-440f-995d-ee82d62d7dc5 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:24.488621 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f552e063-422f-440f-995d-ee82d62d7dc5 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:24.490093 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 393/777] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:00:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 12:00:24.494779 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e121fe17-f733-45c3-b415-ad858fde70a6 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:24.495085 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e121fe17-f733-45c3-b415-ad858fde70a6 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:24.495359 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e121fe17-f733-45c3-b415-ad858fde70a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:24.520556 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-e121fe17-f733-45c3-b415-ad858fde70a6 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:00:24.521827 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-e121fe17-f733-45c3-b415-ad858fde70a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:24.608654 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68ff74f-71e3-4c93-9639-aefd6c2691c4 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:24.632037 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 394/778] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:00:24 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 12:00:24.633462 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e121fe17-f733-45c3-b415-ad858fde70a6 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:24.660362 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6900335b-52d3-4c48-8907-2e1594790a9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:24.661481 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6900335b-52d3-4c48-8907-2e1594790a9c None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:24.662251 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6900335b-52d3-4c48-8907-2e1594790a9c None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:24.663241 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6900335b-52d3-4c48-8907-2e1594790a9c None admin] PATH_INFO: `/v3/services` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:24.664340 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6900335b-52d3-4c48-8907-2e1594790a9c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:24.666864 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6900335b-52d3-4c48-8907-2e1594790a9c None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:24.674100 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 385/779] 10.222.0.49 () {56 vars in 1087 bytes} [Fri Sep 9 12:00:24 2022] GET /identity/v3/services => generated 772 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:24.676266 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6900335b-52d3-4c48-8907-2e1594790a9c None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:24.760048 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e121fe17-f733-45c3-b415-ad858fde70a6 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:24.764896 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:24.781713 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:24.825866 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6900335b-52d3-4c48-8907-2e1594790a9c None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:24.832625 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d79b81-56c4-433b-81b9-5ad3a79bdec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:24.833283 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d79b81-56c4-433b-81b9-5ad3a79bdec5 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:24.833558 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d79b81-56c4-433b-81b9-5ad3a79bdec5 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:24.833827 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d79b81-56c4-433b-81b9-5ad3a79bdec5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:24.857628 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d79b81-56c4-433b-81b9-5ad3a79bdec5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:00:24.864398 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-85d79b81-56c4-433b-81b9-5ad3a79bdec5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 12:00:24.864398 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:24.864398 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:24.864398 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:24.864398 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:24.864398 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:24.864398 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:24.864398 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:24.864398 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:24.864398 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:24.864398 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:24.864398 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:24.864398 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:00:24.864398 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:00:24.864398 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:00:24.864398 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:00:24.865420 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:00:24.865420 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:00:24.865420 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:00:24.865420 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:00:24.865420 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:24.865420 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:24.865420 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:24.865420 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:24.865420 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:24.865420 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:24.865420 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:24.865420 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:24.865420 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:24.865420 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:24.865420 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:24.865420 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:24.866267 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:24.866267 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:24.866267 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:24.866267 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:24.866267 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:00:24.866267 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:00:24.866267 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:00:24.866267 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:00:24.866267 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:00:24.866267 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:00:24.866267 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 12:00:24.866267 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:00:24.880918 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 395/780] 10.222.0.49 () {56 vars in 1102 bytes} [Fri Sep 9 12:00:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:24.884175 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d79b81-56c4-433b-81b9-5ad3a79bdec5 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:24.915857 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d621415-33ad-48e2-a60b-abbe384b786c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:24.917138 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d621415-33ad-48e2-a60b-abbe384b786c None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:24.917668 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d621415-33ad-48e2-a60b-abbe384b786c None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:24.918286 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d621415-33ad-48e2-a60b-abbe384b786c None admin] PATH_INFO: `/v3/services/network` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:24.919107 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d621415-33ad-48e2-a60b-abbe384b786c None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:24.920749 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d621415-33ad-48e2-a60b-abbe384b786c None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:24.935690 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-3d621415-33ad-48e2-a60b-abbe384b786c None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 12:00:24.935690 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:24.935690 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 12:00:24.935690 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 12:00:24.935690 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 12:00:24.935690 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 12:00:24.935690 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 12:00:24.935690 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 12:00:24.935690 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 12:00:24.935690 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:00:24.935690 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 12:00:24.935690 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:00:24.935690 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:24.935690 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:24.935690 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:24.935690 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:24.935690 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:24.936796 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:24.936796 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:24.936796 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:24.936796 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:24.936796 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:24.936796 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:24.936796 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 12:00:24.936796 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 12:00:24.936796 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 12:00:24.936796 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 12:00:24.936796 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:24.936796 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:24.936796 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:24.936796 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:24.936796 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:24.936796 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:24.937637 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:24.937637 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:24.937637 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:24.937637 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:24.937637 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:24.937637 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:24.937637 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:24.937637 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:24.937637 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:24.937637 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:24.937637 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 12:00:24.937637 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 12:00:24.937637 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 12:00:24.937637 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:00:24.937637 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 386/781] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 9 12:00:24 2022] GET /identity/v3/services/network => generated 88 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:24.944295 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d621415-33ad-48e2-a60b-abbe384b786c None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:25.021579 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d79b81-56c4-433b-81b9-5ad3a79bdec5 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:25.058850 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d1fe74-561f-4377-a13d-3c6f27b2e51f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:25.059873 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d1fe74-561f-4377-a13d-3c6f27b2e51f None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:25.060357 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d1fe74-561f-4377-a13d-3c6f27b2e51f None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:25.060838 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d1fe74-561f-4377-a13d-3c6f27b2e51f None admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:25.061703 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d1fe74-561f-4377-a13d-3c6f27b2e51f None admin] RBAC: Adding query filter params (name=service) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:25.062208 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d1fe74-561f-4377-a13d-3c6f27b2e51f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:25.063855 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d1fe74-561f-4377-a13d-3c6f27b2e51f None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:25.072485 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d621415-33ad-48e2-a60b-abbe384b786c None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:25.073527 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 396/782] 10.222.0.49 () {56 vars in 1103 bytes} [Fri Sep 9 12:00:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:25.078111 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d1fe74-561f-4377-a13d-3c6f27b2e51f None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:25.110470 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:25.118261 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:25.119401 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cb5951-d5cc-41d1-a04d-fc64e8261f73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:25.120899 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cb5951-d5cc-41d1-a04d-fc64e8261f73 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:25.121195 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cb5951-d5cc-41d1-a04d-fc64e8261f73 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:25.121817 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cb5951-d5cc-41d1-a04d-fc64e8261f73 None admin] PATH_INFO: `/v3/services` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:25.122814 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cb5951-d5cc-41d1-a04d-fc64e8261f73 None admin] RBAC: Adding query filter params (name=network) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:25.123197 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cb5951-d5cc-41d1-a04d-fc64e8261f73 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:25.124573 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cb5951-d5cc-41d1-a04d-fc64e8261f73 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:25.130743 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 387/783] 10.222.0.49 () {56 vars in 1112 bytes} [Fri Sep 9 12:00:24 2022] GET /identity/v3/services?name=network => generated 124 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:25.132065 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cb5951-d5cc-41d1-a04d-fc64e8261f73 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:25.204940 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d1fe74-561f-4377-a13d-3c6f27b2e51f None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:25.250838 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cb5951-d5cc-41d1-a04d-fc64e8261f73 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:25.251401 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba6e46b-dea6-4f2d-ba0c-5aa10a91c16e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:25.252037 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba6e46b-dea6-4f2d-ba0c-5aa10a91c16e None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:25.252316 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba6e46b-dea6-4f2d-ba0c-5aa10a91c16e None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:25.252587 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba6e46b-dea6-4f2d-ba0c-5aa10a91c16e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:25.270166 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba6e46b-dea6-4f2d-ba0c-5aa10a91c16e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:00:25.276769 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-aba6e46b-dea6-4f2d-ba0c-5aa10a91c16e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 12:00:25.276769 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:25.276769 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:25.276769 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:25.276769 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:25.276769 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:25.276769 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:25.276769 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:25.276769 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:25.276769 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:25.276769 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:25.276769 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:25.276769 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:00:25.276769 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:00:25.276769 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:00:25.276769 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:00:25.277740 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:00:25.277740 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:00:25.277740 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:00:25.277740 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:00:25.277740 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:25.277740 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:25.277740 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:25.277740 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:25.277740 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:25.277740 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:25.277740 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:25.277740 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:25.277740 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:25.277740 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:25.277740 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:25.277740 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:25.278586 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:25.278586 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:25.278586 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:25.278586 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:25.278586 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:00:25.278586 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:00:25.278586 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:00:25.278586 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:00:25.278586 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:00:25.278586 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:00:25.278586 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 12:00:25.278586 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:00:25.279262 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 397/784] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Sep 9 12:00:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:25.279822 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba6e46b-dea6-4f2d-ba0c-5aa10a91c16e None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:25.296772 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2796f7-2646-47dc-b9f2-c6e1c88ddc62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:25.297314 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2796f7-2646-47dc-b9f2-c6e1c88ddc62 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:25.297593 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2796f7-2646-47dc-b9f2-c6e1c88ddc62 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:25.297866 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2796f7-2646-47dc-b9f2-c6e1c88ddc62 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:25.311544 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2796f7-2646-47dc-b9f2-c6e1c88ddc62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:25.313836 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2796f7-2646-47dc-b9f2-c6e1c88ddc62 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:25.333283 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-5b2796f7-2646-47dc-b9f2-c6e1c88ddc62 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:00:25.333283 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:25.333283 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:25.333283 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:25.333283 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:25.333283 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:25.333283 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:25.333283 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:25.333283 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:25.333283 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:25.333283 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:25.333283 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:25.333283 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:00:25.333283 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:00:25.333283 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:00:25.333283 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:00:25.334307 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:25.334307 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:25.334307 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:25.334307 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:25.334307 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:25.334307 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:25.334307 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:25.334307 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:25.334307 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:25.334307 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:25.334307 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:25.334307 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:25.334307 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:25.334307 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:25.334307 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:25.334307 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:25.334307 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:00:25.335225 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:00:25.335225 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:00:25.335225 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:00:25.339263 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 388/785] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 12:00:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:25.340742 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2796f7-2646-47dc-b9f2-c6e1c88ddc62 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:25.401342 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:25.409181 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba6e46b-dea6-4f2d-ba0c-5aa10a91c16e None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:25.410781 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:25.451443 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82ca4e3-1c2b-4d4a-92f4-5107c26c76d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:25.452258 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82ca4e3-1c2b-4d4a-92f4-5107c26c76d1 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:25.452899 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82ca4e3-1c2b-4d4a-92f4-5107c26c76d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:25.453078 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82ca4e3-1c2b-4d4a-92f4-5107c26c76d1 None admin] PATH_INFO: `/v3/services` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:25.453816 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82ca4e3-1c2b-4d4a-92f4-5107c26c76d1 None admin] RBAC: Adding query filter params (type=network) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:25.454217 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82ca4e3-1c2b-4d4a-92f4-5107c26c76d1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:25.455871 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82ca4e3-1c2b-4d4a-92f4-5107c26c76d1 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:25.462520 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 398/786] 10.222.0.49 () {56 vars in 1112 bytes} [Fri Sep 9 12:00:25 2022] GET /identity/v3/services?type=network => generated 350 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:25.463820 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82ca4e3-1c2b-4d4a-92f4-5107c26c76d1 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:25.485980 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2796f7-2646-47dc-b9f2-c6e1c88ddc62 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:25.522145 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119dd825-87c4-4be0-81dd-6b6e0b449fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:25.523030 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119dd825-87c4-4be0-81dd-6b6e0b449fb0 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:25.523145 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119dd825-87c4-4be0-81dd-6b6e0b449fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:25.523449 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119dd825-87c4-4be0-81dd-6b6e0b449fb0 None admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:25.524156 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119dd825-87c4-4be0-81dd-6b6e0b449fb0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:25.524459 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119dd825-87c4-4be0-81dd-6b6e0b449fb0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:25.526043 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119dd825-87c4-4be0-81dd-6b6e0b449fb0 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:25.539275 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 389/787] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 9 12:00:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:25.540202 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119dd825-87c4-4be0-81dd-6b6e0b449fb0 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:25.605831 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82ca4e3-1c2b-4d4a-92f4-5107c26c76d1 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:25.658040 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4014ee34-b0c8-40ed-844e-49af51644031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:25.658790 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4014ee34-b0c8-40ed-844e-49af51644031 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:25.659221 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4014ee34-b0c8-40ed-844e-49af51644031 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:25.660080 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4014ee34-b0c8-40ed-844e-49af51644031 None admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:25.660819 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4014ee34-b0c8-40ed-844e-49af51644031 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:25.661427 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4014ee34-b0c8-40ed-844e-49af51644031 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:25.662675 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4014ee34-b0c8-40ed-844e-49af51644031 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:25.664758 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119dd825-87c4-4be0-81dd-6b6e0b449fb0 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:25.680055 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 399/788] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 9 12:00:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:25.680055 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4014ee34-b0c8-40ed-844e-49af51644031 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:25.682710 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:25.698864 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:25.723177 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb505f3-4244-4579-b008-9e79fc578e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:25.724041 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb505f3-4244-4579-b008-9e79fc578e2b None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:25.724469 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb505f3-4244-4579-b008-9e79fc578e2b None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:25.724887 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb505f3-4244-4579-b008-9e79fc578e2b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:25.726483 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb505f3-4244-4579-b008-9e79fc578e2b None admin] RBAC: Adding query filter params (interface=public, service_id=5cf0d3bbb72d47c78eeeaea8b4db4e8a, region_id=RegionOne) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:25.726967 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb505f3-4244-4579-b008-9e79fc578e2b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:25.728965 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb505f3-4244-4579-b008-9e79fc578e2b None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:25.740610 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 390/789] 10.222.0.49 () {56 vars in 1251 bytes} [Fri Sep 9 12:00:25 2022] GET /identity/v3/endpoints?service_id=5cf0d3bbb72d47c78eeeaea8b4db4e8a&interface=public®ion_id=RegionOne => generated 194 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:25.741923 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb505f3-4244-4579-b008-9e79fc578e2b None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:25.791906 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4014ee34-b0c8-40ed-844e-49af51644031 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:25.830471 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df552d8-ba9a-4e9d-80a5-882070b84d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:25.831136 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df552d8-ba9a-4e9d-80a5-882070b84d14 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:25.831726 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df552d8-ba9a-4e9d-80a5-882070b84d14 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:25.832077 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df552d8-ba9a-4e9d-80a5-882070b84d14 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:25.843123 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df552d8-ba9a-4e9d-80a5-882070b84d14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:25.844755 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df552d8-ba9a-4e9d-80a5-882070b84d14 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:25.861701 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-5df552d8-ba9a-4e9d-80a5-882070b84d14 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:00:25.861701 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:25.861701 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:25.861701 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:25.861701 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:25.861701 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:25.861701 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:25.861701 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:25.861701 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:25.861701 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:25.861701 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:25.861701 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:25.861701 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:00:25.861701 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:00:25.861701 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:00:25.861701 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:00:25.862987 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:25.862987 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:25.862987 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:25.862987 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:25.862987 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:25.862987 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:25.862987 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:25.862987 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:25.862987 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:25.862987 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:25.862987 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:25.862987 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:25.862987 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:25.862987 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:25.862987 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:25.862987 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:25.862987 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:00:25.864281 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:00:25.864281 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:00:25.864281 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:00:25.869843 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 400/790] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 12:00:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:25.870813 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df552d8-ba9a-4e9d-80a5-882070b84d14 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:25.881680 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb505f3-4244-4579-b008-9e79fc578e2b None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:25.921968 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d221b1db-ee73-45b4-8435-0e80603acdde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:25.922885 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d221b1db-ee73-45b4-8435-0e80603acdde None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:25.923368 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d221b1db-ee73-45b4-8435-0e80603acdde None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:25.923792 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d221b1db-ee73-45b4-8435-0e80603acdde None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:25.933635 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d221b1db-ee73-45b4-8435-0e80603acdde None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:25.935554 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d221b1db-ee73-45b4-8435-0e80603acdde None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:25.948886 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-d221b1db-ee73-45b4-8435-0e80603acdde None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 12:00:25.948886 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:25.948886 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:25.948886 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:25.948886 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:25.948886 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:25.948886 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:25.948886 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:25.948886 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:25.948886 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:25.948886 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:25.948886 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:25.948886 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 12:00:25.948886 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 12:00:25.948886 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 12:00:25.948886 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 12:00:25.949997 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:25.949997 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:25.949997 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 12:00:25.949997 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:00:25.949997 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 12:00:25.949997 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:00:25.949997 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:25.949997 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:25.949997 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:25.949997 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:25.949997 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:25.949997 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:25.949997 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:25.949997 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:25.949997 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:25.949997 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:25.949997 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:25.949997 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:25.950979 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:25.950979 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:25.950979 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 12:00:25.950979 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 12:00:25.950979 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 12:00:25.950979 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 12:00:25.950979 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 12:00:25.950979 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 12:00:25.950979 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 12:00:25.950979 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:00:25.953629 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 391/791] 10.222.0.49 () {56 vars in 1117 bytes} [Fri Sep 9 12:00:25 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:25.959996 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d221b1db-ee73-45b4-8435-0e80603acdde None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:25.996784 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:26.069488 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d221b1db-ee73-45b4-8435-0e80603acdde None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:26.087188 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:26.110492 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6b8b6c-00bb-47c6-ac6a-e7c8dd65a543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:26.111459 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6b8b6c-00bb-47c6-ac6a-e7c8dd65a543 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:26.112501 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6b8b6c-00bb-47c6-ac6a-e7c8dd65a543 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:26.113034 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6b8b6c-00bb-47c6-ac6a-e7c8dd65a543 None admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:26.114166 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6b8b6c-00bb-47c6-ac6a-e7c8dd65a543 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:26.114657 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6b8b6c-00bb-47c6-ac6a-e7c8dd65a543 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:26.116953 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6b8b6c-00bb-47c6-ac6a-e7c8dd65a543 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:26.125391 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df552d8-ba9a-4e9d-80a5-882070b84d14 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:26.132311 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 392/792] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 9 12:00:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:26.133433 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6b8b6c-00bb-47c6-ac6a-e7c8dd65a543 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:26.170562 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6169b24-a85d-481e-86cb-27f52ed411d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:26.171375 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6169b24-a85d-481e-86cb-27f52ed411d2 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:26.171652 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6169b24-a85d-481e-86cb-27f52ed411d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:26.171922 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6169b24-a85d-481e-86cb-27f52ed411d2 None admin] PATH_INFO: `/v3/services` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:26.172629 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6169b24-a85d-481e-86cb-27f52ed411d2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:26.174069 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6169b24-a85d-481e-86cb-27f52ed411d2 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:26.180412 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 401/793] 10.222.0.49 () {56 vars in 1087 bytes} [Fri Sep 9 12:00:25 2022] GET /identity/v3/services => generated 772 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:26.181882 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6169b24-a85d-481e-86cb-27f52ed411d2 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:26.243273 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:26.255065 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6b8b6c-00bb-47c6-ac6a-e7c8dd65a543 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:26.292865 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6169b24-a85d-481e-86cb-27f52ed411d2 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:26.306859 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf10aba-914a-485e-8313-d990e5f6809d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:26.307516 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf10aba-914a-485e-8313-d990e5f6809d None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:26.307895 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf10aba-914a-485e-8313-d990e5f6809d None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:26.308173 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf10aba-914a-485e-8313-d990e5f6809d None admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:26.309019 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf10aba-914a-485e-8313-d990e5f6809d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:26.309331 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf10aba-914a-485e-8313-d990e5f6809d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:26.310801 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf10aba-914a-485e-8313-d990e5f6809d None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:26.330421 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb52e90-6041-4090-b902-a2e572cd9262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:26.331099 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb52e90-6041-4090-b902-a2e572cd9262 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:26.331407 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb52e90-6041-4090-b902-a2e572cd9262 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:26.331676 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb52e90-6041-4090-b902-a2e572cd9262 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:26.331783 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 393/794] 10.222.0.49 () {56 vars in 1149 bytes} [Fri Sep 9 12:00:26 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:26.333266 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d52ce1f-80ba-47b6-b863-90d4cfb64d3b None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:26.333553 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d52ce1f-80ba-47b6-b863-90d4cfb64d3b None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:26.333999 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:26.334365 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d52ce1f-80ba-47b6-b863-90d4cfb64d3b None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:26.340988 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb52e90-6041-4090-b902-a2e572cd9262 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:26.341312 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 394/795] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:00:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:00:26.342321 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb52e90-6041-4090-b902-a2e572cd9262 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:26.343333 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d52ce1f-80ba-47b6-b863-90d4cfb64d3b None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:26.359078 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-1eb52e90-6041-4090-b902-a2e572cd9262 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 09 12:00:26.359078 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:26.359078 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:26.359078 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:26.359078 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:26.359078 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:26.359078 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:26.359078 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:26.359078 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:26.359078 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:26.359078 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:26.359078 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:26.359078 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 12:00:26.359078 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 12:00:26.359078 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 12:00:26.359078 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 12:00:26.360114 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:26.360114 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:26.360114 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 12:00:26.360114 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:00:26.360114 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 12:00:26.360114 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:00:26.360114 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:26.360114 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:26.360114 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:26.360114 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:26.360114 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:26.360114 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:26.360114 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:26.360114 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:26.360114 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:26.360114 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:26.360114 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:26.360114 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:26.361077 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:26.361077 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:26.361077 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 12:00:26.361077 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 12:00:26.361077 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 12:00:26.361077 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 12:00:26.361077 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 12:00:26.361077 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 12:00:26.361077 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 09 12:00:26.361077 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:00:26.361077 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 402/796] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Sep 9 12:00:26 2022] GET /identity/v3/users/swift => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:26.366470 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13df95e-7e4c-4325-ba26-2da38173e383 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:26.366778 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13df95e-7e4c-4325-ba26-2da38173e383 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:26.367058 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13df95e-7e4c-4325-ba26-2da38173e383 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:26.387908 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-d13df95e-7e4c-4325-ba26-2da38173e383 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:00:26.389394 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-d13df95e-7e4c-4325-ba26-2da38173e383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:26.480080 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d52ce1f-80ba-47b6-b863-90d4cfb64d3b None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:26.481608 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 403/797] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:00:26 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 12:00:26.482692 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13df95e-7e4c-4325-ba26-2da38173e383 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:26.576762 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234a5f60-6afb-4e45-9669-aa2f4fcdd439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:26.577814 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234a5f60-6afb-4e45-9669-aa2f4fcdd439 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:26.578295 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234a5f60-6afb-4e45-9669-aa2f4fcdd439 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:26.578798 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234a5f60-6afb-4e45-9669-aa2f4fcdd439 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:26.597507 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13df95e-7e4c-4325-ba26-2da38173e383 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:26.603953 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234a5f60-6afb-4e45-9669-aa2f4fcdd439 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:26.605688 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234a5f60-6afb-4e45-9669-aa2f4fcdd439 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:26.623241 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:26.637834 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4e65a5-2f05-4169-a52c-d07966227a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:26.638707 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4e65a5-2f05-4169-a52c-d07966227a5f None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:26.639152 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4e65a5-2f05-4169-a52c-d07966227a5f None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:26.639570 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4e65a5-2f05-4169-a52c-d07966227a5f None admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:26.640624 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4e65a5-2f05-4169-a52c-d07966227a5f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:26.641209 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4e65a5-2f05-4169-a52c-d07966227a5f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:26.642799 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4e65a5-2f05-4169-a52c-d07966227a5f None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:26.649666 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-234a5f60-6afb-4e45-9669-aa2f4fcdd439 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:00:26.649666 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:26.649666 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:26.649666 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:26.649666 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:26.649666 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:26.649666 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:26.649666 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:26.649666 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:26.649666 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:26.649666 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:26.649666 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:26.649666 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:00:26.649666 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:00:26.649666 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:00:26.649666 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:00:26.650631 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:26.650631 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:26.650631 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:26.650631 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:26.650631 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:26.650631 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:26.650631 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:26.650631 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:26.650631 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:26.650631 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:26.650631 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:26.650631 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:26.650631 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:26.650631 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:26.650631 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:26.650631 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:26.650631 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:00:26.651566 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:00:26.651566 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:00:26.651566 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:00:26.660911 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 395/798] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 12:00:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:26.665825 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 404/799] 10.222.0.49 () {56 vars in 1135 bytes} [Fri Sep 9 12:00:26 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:26.666231 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234a5f60-6afb-4e45-9669-aa2f4fcdd439 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:26.674495 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:26.676581 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4e65a5-2f05-4169-a52c-d07966227a5f None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:26.828986 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234a5f60-6afb-4e45-9669-aa2f4fcdd439 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:26.834014 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4e65a5-2f05-4169-a52c-d07966227a5f None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:26.866831 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3d7354-e52f-48d9-a42d-21c7cc29a229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:26.868091 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3d7354-e52f-48d9-a42d-21c7cc29a229 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:26.868672 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3d7354-e52f-48d9-a42d-21c7cc29a229 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:26.869240 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3d7354-e52f-48d9-a42d-21c7cc29a229 None admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:26.869549 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b173dfa-0c02-4a82-bb39-f5656c41105f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:26.870177 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b173dfa-0c02-4a82-bb39-f5656c41105f None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:26.870453 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b173dfa-0c02-4a82-bb39-f5656c41105f None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:26.870721 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b173dfa-0c02-4a82-bb39-f5656c41105f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:26.870976 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3d7354-e52f-48d9-a42d-21c7cc29a229 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:26.871668 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3d7354-e52f-48d9-a42d-21c7cc29a229 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:26.873459 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3d7354-e52f-48d9-a42d-21c7cc29a229 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:26.887354 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 396/800] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 9 12:00:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:00:26.895203 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b173dfa-0c02-4a82-bb39-f5656c41105f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:26.895203 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b173dfa-0c02-4a82-bb39-f5656c41105f None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:26.896312 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3d7354-e52f-48d9-a42d-21c7cc29a229 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:26.922479 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-1b173dfa-0c02-4a82-bb39-f5656c41105f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:00:26.922479 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:26.922479 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:26.922479 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:26.922479 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:26.922479 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:26.922479 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:26.922479 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:26.922479 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:26.922479 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:26.922479 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:26.922479 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:26.922479 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:00:26.922479 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:00:26.922479 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:00:26.922479 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:00:26.929714 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:26.929714 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:26.929714 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:26.929714 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:26.929714 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:26.929714 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:26.929714 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:26.929714 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:26.929714 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:26.929714 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:26.929714 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:26.929714 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:26.929714 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:26.929714 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:26.929714 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:26.929714 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:26.929714 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:00:26.935262 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:00:26.935262 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:00:26.935262 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:00:26.935262 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 405/801] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 12:00:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:26.935262 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b173dfa-0c02-4a82-bb39-f5656c41105f None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:26.974534 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:27.001155 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:27.047235 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3d7354-e52f-48d9-a42d-21c7cc29a229 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:27.061062 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b173dfa-0c02-4a82-bb39-f5656c41105f None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:27.085905 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3017e393-f02b-4b8a-9068-6388db53c7dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:27.086467 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3017e393-f02b-4b8a-9068-6388db53c7dc None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:27.086750 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3017e393-f02b-4b8a-9068-6388db53c7dc None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:27.087056 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3017e393-f02b-4b8a-9068-6388db53c7dc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:27.093923 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3017e393-f02b-4b8a-9068-6388db53c7dc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:27.095422 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3017e393-f02b-4b8a-9068-6388db53c7dc None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:27.101561 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30b4873-7992-4b2b-9ced-6e18caf3cfbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:27.102203 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30b4873-7992-4b2b-9ced-6e18caf3cfbf None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:27.102486 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30b4873-7992-4b2b-9ced-6e18caf3cfbf None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:27.102756 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30b4873-7992-4b2b-9ced-6e18caf3cfbf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:27.108802 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-3017e393-f02b-4b8a-9068-6388db53c7dc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:00:27.108802 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:27.108802 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:27.108802 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:27.108802 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:27.108802 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:27.108802 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:27.108802 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:27.108802 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:27.108802 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:27.108802 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:27.108802 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:27.108802 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 12:00:27.108802 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 12:00:27.108802 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 12:00:27.109658 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 12:00:27.109658 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:27.109658 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:27.109658 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:27.109658 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:27.109658 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:27.109658 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:27.109658 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:27.109658 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:27.109658 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:27.109658 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:27.109658 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:27.109658 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:27.109658 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:27.109658 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:27.109658 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:27.109658 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:27.110553 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 12:00:27.110553 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 12:00:27.110553 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 12:00:27.110553 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 12:00:27.110553 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 12:00:27.110553 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 12:00:27.110553 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:00:27.110553 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:00:27.110553 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 397/802] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 9 12:00:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:27.111069 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3017e393-f02b-4b8a-9068-6388db53c7dc None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:27.118430 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30b4873-7992-4b2b-9ced-6e18caf3cfbf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:00:27.123736 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-f30b4873-7992-4b2b-9ced-6e18caf3cfbf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 12:00:27.123736 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:27.123736 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:27.123736 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:27.123736 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:27.123736 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:27.123736 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:27.123736 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:27.123736 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:27.123736 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:27.123736 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:27.123736 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:27.123736 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:00:27.123736 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:00:27.123736 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:00:27.123736 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:00:27.124865 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:00:27.124865 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:00:27.124865 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:00:27.124865 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:00:27.124865 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:27.124865 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:27.124865 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:27.124865 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:27.124865 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:27.124865 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:27.124865 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:27.124865 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:27.124865 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:27.124865 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:27.124865 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:27.124865 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:27.126037 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:27.126037 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:27.126037 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:27.126037 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:27.126037 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:00:27.126037 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:00:27.126037 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:00:27.126037 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:00:27.126037 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:00:27.126037 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:00:27.126037 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 12:00:27.126037 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:00:27.126037 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 406/803] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Sep 9 12:00:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:27.131534 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acab5c73-ee29-4967-93a4-d2a49d44cbc5 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:27.131818 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acab5c73-ee29-4967-93a4-d2a49d44cbc5 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:27.132095 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acab5c73-ee29-4967-93a4-d2a49d44cbc5 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:27.133365 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 407/804] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:00:27 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:00:27.134054 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acab5c73-ee29-4967-93a4-d2a49d44cbc5 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:27.223425 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3017e393-f02b-4b8a-9068-6388db53c7dc None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:27.238705 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:27.247209 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:27.249332 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acab5c73-ee29-4967-93a4-d2a49d44cbc5 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:27.260539 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cac0035-573a-4df4-8dc5-61015fca3d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:27.261247 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cac0035-573a-4df4-8dc5-61015fca3d19 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:27.261527 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cac0035-573a-4df4-8dc5-61015fca3d19 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:27.261804 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cac0035-573a-4df4-8dc5-61015fca3d19 None admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:27.262449 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cac0035-573a-4df4-8dc5-61015fca3d19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:27.262754 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cac0035-573a-4df4-8dc5-61015fca3d19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:27.264221 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cac0035-573a-4df4-8dc5-61015fca3d19 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:27.277921 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 398/805] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 9 12:00:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:27.278800 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cac0035-573a-4df4-8dc5-61015fca3d19 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:27.295992 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859461d9-42d4-46f6-a45d-8b373e347e7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:27.296983 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859461d9-42d4-46f6-a45d-8b373e347e7c None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:27.297275 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859461d9-42d4-46f6-a45d-8b373e347e7c None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:27.297546 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859461d9-42d4-46f6-a45d-8b373e347e7c None admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:27.298500 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859461d9-42d4-46f6-a45d-8b373e347e7c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:27.298803 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859461d9-42d4-46f6-a45d-8b373e347e7c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:27.300403 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859461d9-42d4-46f6-a45d-8b373e347e7c None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:27.320845 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 408/806] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Sep 9 12:00:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:27.320845 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d864f120-0fdd-4d9f-9625-3119deb2c307 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:27.320845 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d864f120-0fdd-4d9f-9625-3119deb2c307 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:27.320845 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d864f120-0fdd-4d9f-9625-3119deb2c307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:27.339629 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-d864f120-0fdd-4d9f-9625-3119deb2c307 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:00:27.340752 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-d864f120-0fdd-4d9f-9625-3119deb2c307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:27.393575 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cac0035-573a-4df4-8dc5-61015fca3d19 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:27.412358 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 409/807] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:00:27 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 12:00:27.413348 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d864f120-0fdd-4d9f-9625-3119deb2c307 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:27.442996 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07e3768-c53a-4457-85a0-873d498f7121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:27.444307 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07e3768-c53a-4457-85a0-873d498f7121 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:27.444762 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07e3768-c53a-4457-85a0-873d498f7121 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:27.445203 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07e3768-c53a-4457-85a0-873d498f7121 None admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:27.446030 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07e3768-c53a-4457-85a0-873d498f7121 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:27.447039 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07e3768-c53a-4457-85a0-873d498f7121 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:27.449152 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07e3768-c53a-4457-85a0-873d498f7121 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:27.462257 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 399/808] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 9 12:00:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:27.463277 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07e3768-c53a-4457-85a0-873d498f7121 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:27.522540 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:27.523689 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:27.543256 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d864f120-0fdd-4d9f-9625-3119deb2c307 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:27.582171 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5868f32-65d5-4897-9999-b3536a31d73e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:27.582815 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5868f32-65d5-4897-9999-b3536a31d73e None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:27.583116 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5868f32-65d5-4897-9999-b3536a31d73e None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:27.583382 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5868f32-65d5-4897-9999-b3536a31d73e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:27.591602 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5868f32-65d5-4897-9999-b3536a31d73e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:27.593068 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5868f32-65d5-4897-9999-b3536a31d73e None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:27.602938 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07e3768-c53a-4457-85a0-873d498f7121 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:27.611771 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-a5868f32-65d5-4897-9999-b3536a31d73e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:00:27.611771 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:27.611771 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:27.611771 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:27.611771 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:27.611771 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:27.611771 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:27.611771 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:27.611771 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:27.611771 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:27.611771 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:27.611771 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:27.611771 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 12:00:27.611771 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 12:00:27.611771 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 12:00:27.613681 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 12:00:27.613681 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:27.613681 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:27.613681 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:27.613681 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:27.613681 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:27.613681 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:27.613681 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:27.613681 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:27.613681 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:27.613681 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:27.613681 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:27.613681 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:27.613681 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:27.613681 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:27.613681 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:27.613681 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:27.614629 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 12:00:27.614629 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 12:00:27.614629 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 12:00:27.614629 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 12:00:27.614629 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 12:00:27.614629 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 12:00:27.614629 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:00:27.614629 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:00:27.618872 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 410/809] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 9 12:00:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:27.620043 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5868f32-65d5-4897-9999-b3536a31d73e None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:27.657288 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314ac6ea-bcd2-43e3-8b43-a0610dcad024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:27.658224 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314ac6ea-bcd2-43e3-8b43-a0610dcad024 None admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:27.658730 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314ac6ea-bcd2-43e3-8b43-a0610dcad024 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:27.659276 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314ac6ea-bcd2-43e3-8b43-a0610dcad024 None admin] PATH_INFO: `/v3/projects/fe863fcd75f1406ebbffc5016078f73e/users/e368b4e668c54e25bff80a8e8a8c3588/roles/8abeff2cb87d4b7fa72b27b028005877` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:27.677059 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314ac6ea-bcd2-43e3-8b43-a0610dcad024 None admin] RBAC: Authorizing `identity:create_grant(project_id=fe863fcd75f1406ebbffc5016078f73e, user_id=e368b4e668c54e25bff80a8e8a8c3588, role_id=8abeff2cb87d4b7fa72b27b028005877)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:27.678833 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314ac6ea-bcd2-43e3-8b43-a0610dcad024 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:27.702485 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 400/810] 10.222.0.49 () {58 vars in 1439 bytes} [Fri Sep 9 12:00:27 2022] PUT /identity/v3/projects/fe863fcd75f1406ebbffc5016078f73e/users/e368b4e668c54e25bff80a8e8a8c3588/roles/8abeff2cb87d4b7fa72b27b028005877 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:00:27.703861 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314ac6ea-bcd2-43e3-8b43-a0610dcad024 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:27.731260 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5868f32-65d5-4897-9999-b3536a31d73e None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:27.778379 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5398382a-079b-4fb3-a852-a70556e1a1b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:27.779671 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5398382a-079b-4fb3-a852-a70556e1a1b7 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:27.780585 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5398382a-079b-4fb3-a852-a70556e1a1b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:27.781034 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5398382a-079b-4fb3-a852-a70556e1a1b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:27.800345 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5398382a-079b-4fb3-a852-a70556e1a1b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:27.802614 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5398382a-079b-4fb3-a852-a70556e1a1b7 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:27.807034 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:27.825733 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-5398382a-079b-4fb3-a852-a70556e1a1b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:00:27.825733 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:27.825733 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:27.825733 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:27.825733 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:27.825733 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:27.825733 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:27.825733 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:27.825733 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:27.825733 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:27.825733 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:27.825733 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:27.825733 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:00:27.825733 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:00:27.825733 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:00:27.825733 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:00:27.826764 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:27.826764 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:27.826764 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:27.826764 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:27.826764 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:27.826764 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:27.826764 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:27.826764 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:27.826764 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:27.826764 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:27.826764 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:27.826764 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:27.826764 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:27.826764 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:27.826764 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:27.826764 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:27.826764 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:00:27.827711 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:00:27.827711 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:00:27.827711 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:00:27.828770 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 411/811] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 12:00:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:27.829594 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5398382a-079b-4fb3-a852-a70556e1a1b7 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:27.835703 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:27.845453 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314ac6ea-bcd2-43e3-8b43-a0610dcad024 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:27.894064 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6d4524-c4db-4a4a-be69-4d16f2f06e40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:27.895474 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6d4524-c4db-4a4a-be69-4d16f2f06e40 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:27.895954 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6d4524-c4db-4a4a-be69-4d16f2f06e40 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:27.896460 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6d4524-c4db-4a4a-be69-4d16f2f06e40 None admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:27.897163 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6d4524-c4db-4a4a-be69-4d16f2f06e40 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:27.897467 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6d4524-c4db-4a4a-be69-4d16f2f06e40 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:27.898917 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6d4524-c4db-4a4a-be69-4d16f2f06e40 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:27.912519 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 401/812] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Sep 9 12:00:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:27.913180 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6d4524-c4db-4a4a-be69-4d16f2f06e40 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:27.973825 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5398382a-079b-4fb3-a852-a70556e1a1b7 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:28.020253 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6d4524-c4db-4a4a-be69-4d16f2f06e40 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:28.020637 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0d52cd-733d-4853-9770-05f83cab38aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:28.021276 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0d52cd-733d-4853-9770-05f83cab38aa None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:28.021546 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0d52cd-733d-4853-9770-05f83cab38aa None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:28.022070 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0d52cd-733d-4853-9770-05f83cab38aa None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:28.023573 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0d52cd-733d-4853-9770-05f83cab38aa None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:28.024939 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0d52cd-733d-4853-9770-05f83cab38aa None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:28.037052 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-fe0d52cd-733d-4853-9770-05f83cab38aa None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 12:00:28.037052 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:28.037052 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 12:00:28.037052 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 12:00:28.037052 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 12:00:28.037052 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 12:00:28.037052 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 12:00:28.037052 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 12:00:28.037052 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 12:00:28.037052 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:00:28.037052 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 12:00:28.037052 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:00:28.037052 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:28.037052 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:28.037052 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:28.037052 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:28.037052 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:28.038085 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:28.038085 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:28.038085 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:28.038085 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:28.038085 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:28.038085 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:28.038085 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 12:00:28.038085 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 12:00:28.038085 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 12:00:28.038085 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 12:00:28.038085 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:28.038085 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:28.038085 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:28.038085 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:28.038085 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:28.038085 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:28.039007 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:28.039007 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:28.039007 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:28.039007 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:28.039007 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:28.039007 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:28.039007 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:28.039007 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:28.039007 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:28.039007 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:28.039007 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 12:00:28.039007 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 12:00:28.039007 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 12:00:28.039007 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:00:28.039760 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 412/813] 10.222.0.49 () {56 vars in 1129 bytes} [Fri Sep 9 12:00:27 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:28.039827 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0d52cd-733d-4853-9770-05f83cab38aa None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:28.057923 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e781f42-988a-4c93-a390-3f121fccb8a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:28.058530 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e781f42-988a-4c93-a390-3f121fccb8a7 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:28.058806 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e781f42-988a-4c93-a390-3f121fccb8a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:28.059118 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e781f42-988a-4c93-a390-3f121fccb8a7 None admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:28.059912 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e781f42-988a-4c93-a390-3f121fccb8a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:28.060311 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e781f42-988a-4c93-a390-3f121fccb8a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:28.061739 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e781f42-988a-4c93-a390-3f121fccb8a7 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:28.076799 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 402/814] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 9 12:00:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:28.078025 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e781f42-988a-4c93-a390-3f121fccb8a7 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:28.097709 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:28.119909 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:28.164476 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0d52cd-733d-4853-9770-05f83cab38aa None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:28.200714 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbab47b9-b4d8-4e6c-a4d5-7822e77f172f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:28.201378 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbab47b9-b4d8-4e6c-a4d5-7822e77f172f None admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:28.201655 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbab47b9-b4d8-4e6c-a4d5-7822e77f172f None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:28.201927 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbab47b9-b4d8-4e6c-a4d5-7822e77f172f None admin] PATH_INFO: `/v3/projects/fe863fcd75f1406ebbffc5016078f73e/users/92e652ba827849b5a3fc1213d04a1b76/roles/3f2ca33902a04b6dbc2ef7d06d7289d3` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:28.204172 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbab47b9-b4d8-4e6c-a4d5-7822e77f172f None admin] RBAC: Authorizing `identity:create_grant(project_id=fe863fcd75f1406ebbffc5016078f73e, user_id=92e652ba827849b5a3fc1213d04a1b76, role_id=3f2ca33902a04b6dbc2ef7d06d7289d3)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:28.205557 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbab47b9-b4d8-4e6c-a4d5-7822e77f172f None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:28.214439 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e781f42-988a-4c93-a390-3f121fccb8a7 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:28.224667 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 413/815] 10.222.0.49 () {58 vars in 1439 bytes} [Fri Sep 9 12:00:28 2022] PUT /identity/v3/projects/fe863fcd75f1406ebbffc5016078f73e/users/92e652ba827849b5a3fc1213d04a1b76/roles/3f2ca33902a04b6dbc2ef7d06d7289d3 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:00:28.226122 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbab47b9-b4d8-4e6c-a4d5-7822e77f172f None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:28.263804 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e7b1c0-c3e2-4d3d-aa20-826b1eeeeb25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:28.264792 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e7b1c0-c3e2-4d3d-aa20-826b1eeeeb25 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:28.264792 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e7b1c0-c3e2-4d3d-aa20-826b1eeeeb25 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:28.264969 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e7b1c0-c3e2-4d3d-aa20-826b1eeeeb25 None admin] PATH_INFO: `/v3/services` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:28.265598 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e7b1c0-c3e2-4d3d-aa20-826b1eeeeb25 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:28.265900 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e7b1c0-c3e2-4d3d-aa20-826b1eeeeb25 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:28.267353 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e7b1c0-c3e2-4d3d-aa20-826b1eeeeb25 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:28.273670 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 403/816] 10.222.0.49 () {56 vars in 1124 bytes} [Fri Sep 9 12:00:28 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:28.278770 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e7b1c0-c3e2-4d3d-aa20-826b1eeeeb25 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:28.365349 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbab47b9-b4d8-4e6c-a4d5-7822e77f172f None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:28.381974 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:28.394145 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:28.407866 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e7b1c0-c3e2-4d3d-aa20-826b1eeeeb25 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:28.415947 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b597163-7ddd-484d-b36a-4b7731bf1c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:28.416955 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b597163-7ddd-484d-b36a-4b7731bf1c23 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:28.417545 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b597163-7ddd-484d-b36a-4b7731bf1c23 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:28.418021 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b597163-7ddd-484d-b36a-4b7731bf1c23 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:28.428903 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b597163-7ddd-484d-b36a-4b7731bf1c23 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:28.431088 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b597163-7ddd-484d-b36a-4b7731bf1c23 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:28.447810 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-5b597163-7ddd-484d-b36a-4b7731bf1c23 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 12:00:28.447810 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:28.447810 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:28.447810 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:28.447810 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:28.447810 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:28.447810 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:28.447810 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:28.447810 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:28.447810 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:28.447810 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:28.447810 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:28.447810 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 12:00:28.447810 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 12:00:28.447810 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 12:00:28.447810 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 12:00:28.448870 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:28.448870 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:28.448870 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 12:00:28.448870 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:00:28.448870 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 12:00:28.448870 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:00:28.448870 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:28.448870 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:28.448870 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:28.448870 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:28.448870 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:28.448870 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:28.448870 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:28.448870 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:28.448870 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:28.448870 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:28.448870 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:28.448870 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:28.449828 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:28.449828 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:28.449828 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 12:00:28.449828 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 12:00:28.449828 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 12:00:28.449828 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 12:00:28.449828 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 12:00:28.449828 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 12:00:28.449828 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 12:00:28.449828 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:00:28.450394 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d0494c-b4c5-4627-b459-0d345eb105ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:28.450394 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d0494c-b4c5-4627-b459-0d345eb105ab None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:28.451161 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d0494c-b4c5-4627-b459-0d345eb105ab None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:28.451161 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d0494c-b4c5-4627-b459-0d345eb105ab None admin] PATH_INFO: `/v3/services` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:28.451583 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d0494c-b4c5-4627-b459-0d345eb105ab None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:28.451881 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d0494c-b4c5-4627-b459-0d345eb105ab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:28.453240 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d0494c-b4c5-4627-b459-0d345eb105ab None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:28.453615 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 414/817] 10.222.0.49 () {56 vars in 1093 bytes} [Fri Sep 9 12:00:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:28.460118 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b597163-7ddd-484d-b36a-4b7731bf1c23 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:28.460862 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 404/818] 10.222.0.49 () {56 vars in 1124 bytes} [Fri Sep 9 12:00:28 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:28.466403 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d0494c-b4c5-4627-b459-0d345eb105ab None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:28.588817 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b597163-7ddd-484d-b36a-4b7731bf1c23 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:28.591086 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d0494c-b4c5-4627-b459-0d345eb105ab None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:28.628470 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61e4655-cee5-4c9a-92b0-722e53ba5ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:28.628470 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61e4655-cee5-4c9a-92b0-722e53ba5ba4 None admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:28.628470 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61e4655-cee5-4c9a-92b0-722e53ba5ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:28.628470 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61e4655-cee5-4c9a-92b0-722e53ba5ba4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:28.629364 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61e4655-cee5-4c9a-92b0-722e53ba5ba4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:28.630922 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61e4655-cee5-4c9a-92b0-722e53ba5ba4 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:28.634492 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2eb59ee-dd8e-4042-bf2c-f29b4768da88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:28.635455 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eb59ee-dd8e-4042-bf2c-f29b4768da88 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:28.635952 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eb59ee-dd8e-4042-bf2c-f29b4768da88 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:28.636426 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eb59ee-dd8e-4042-bf2c-f29b4768da88 None admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:28.637358 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2eb59ee-dd8e-4042-bf2c-f29b4768da88 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:28.637874 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2eb59ee-dd8e-4042-bf2c-f29b4768da88 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:28.639525 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2eb59ee-dd8e-4042-bf2c-f29b4768da88 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:28.663319 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:28.676020 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 405/819] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:00:28 2022] POST /identity/v3/endpoints => generated 346 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:00:28.686448 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afe7d1b-929b-4da9-a033-777f00d4eaf1 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:28.686778 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afe7d1b-929b-4da9-a033-777f00d4eaf1 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:28.687012 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afe7d1b-929b-4da9-a033-777f00d4eaf1 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:28.692332 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 415/821] 10.222.0.49 () {56 vars in 1133 bytes} [Fri Sep 9 12:00:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:00:28.692514 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 406/821] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:00:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:00:28.695728 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2c8153-0d68-42ab-b30c-2b52c065818a None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:28.696010 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2c8153-0d68-42ab-b30c-2b52c065818a None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:28.696275 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2c8153-0d68-42ab-b30c-2b52c065818a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:28.698143 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afe7d1b-929b-4da9-a033-777f00d4eaf1 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:28.733962 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-7e2c8153-0d68-42ab-b30c-2b52c065818a None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:00:28.736401 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-7e2c8153-0d68-42ab-b30c-2b52c065818a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:28.797936 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:28.891133 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 416/822] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:00:28 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 12:00:28.920556 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afe7d1b-929b-4da9-a033-777f00d4eaf1 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:28.956771 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf03d277-d67f-4aae-8861-ca90b786498d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:28.957384 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf03d277-d67f-4aae-8861-ca90b786498d None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:28.957854 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf03d277-d67f-4aae-8861-ca90b786498d None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:28.958130 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf03d277-d67f-4aae-8861-ca90b786498d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:29.026544 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:29.031112 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf03d277-d67f-4aae-8861-ca90b786498d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:29.032620 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf03d277-d67f-4aae-8861-ca90b786498d None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:29.145188 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 407/823] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 9 12:00:28 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 12:00:29.150469 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2c8153-0d68-42ab-b30c-2b52c065818a None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:29.248403 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf03d277-d67f-4aae-8861-ca90b786498d None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:29.277989 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2c8153-0d68-42ab-b30c-2b52c065818a None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:29.346039 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becab7da-cb31-4b6c-b714-9af517b206fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:29.347322 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becab7da-cb31-4b6c-b714-9af517b206fe None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:29.348093 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becab7da-cb31-4b6c-b714-9af517b206fe None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:29.349558 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becab7da-cb31-4b6c-b714-9af517b206fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:29.374011 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becab7da-cb31-4b6c-b714-9af517b206fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:29.375829 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becab7da-cb31-4b6c-b714-9af517b206fe None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:29.400296 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-becab7da-cb31-4b6c-b714-9af517b206fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:00:29.400296 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:29.400296 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:29.400296 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:29.400296 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:29.400296 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:29.400296 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:29.400296 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:29.400296 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:29.400296 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:29.400296 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:29.400296 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:29.400296 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:00:29.400296 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:00:29.400296 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:00:29.400296 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:00:29.401260 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:29.401260 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:29.401260 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:29.401260 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:29.401260 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:29.401260 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:29.401260 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:29.401260 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:29.401260 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:29.401260 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:29.401260 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:29.401260 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:29.401260 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:29.401260 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:29.401260 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:29.401260 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:29.401260 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:00:29.402141 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:00:29.402141 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:00:29.402141 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:00:29.402141 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 417/824] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 12:00:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:29.412120 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becab7da-cb31-4b6c-b714-9af517b206fe None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:29.416131 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf03d277-d67f-4aae-8861-ca90b786498d None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:29.427312 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:29.452880 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:29.489005 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae11e73-c601-4f1f-ad63-4468c1bdd535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:29.489754 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae11e73-c601-4f1f-ad63-4468c1bdd535 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:29.490037 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae11e73-c601-4f1f-ad63-4468c1bdd535 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:29.490308 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae11e73-c601-4f1f-ad63-4468c1bdd535 None admin] PATH_INFO: `/v3/services/network` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:29.490889 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae11e73-c601-4f1f-ad63-4468c1bdd535 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:29.492497 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae11e73-c601-4f1f-ad63-4468c1bdd535 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:29.509053 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-9ae11e73-c601-4f1f-ad63-4468c1bdd535 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 12:00:29.509053 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:29.509053 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 12:00:29.509053 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 12:00:29.509053 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 12:00:29.509053 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 12:00:29.509053 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 12:00:29.509053 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 12:00:29.509053 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 12:00:29.509053 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:00:29.509053 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 12:00:29.509053 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:00:29.509053 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:29.509053 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:29.509053 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:29.509053 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:29.509053 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:29.510105 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:29.510105 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:29.510105 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:29.510105 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:29.510105 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:29.510105 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:29.510105 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 12:00:29.510105 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 12:00:29.510105 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 12:00:29.510105 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 12:00:29.510105 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:29.510105 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:29.510105 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:29.510105 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:29.510105 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:29.510105 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:29.511372 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:29.511372 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:29.511372 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:29.511372 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:29.511372 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:29.511372 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:29.511372 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:29.511372 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:29.511372 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:29.511372 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:29.511372 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 12:00:29.511372 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 12:00:29.511372 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 12:00:29.511372 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:00:29.511372 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 408/825] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 9 12:00:29 2022] GET /identity/v3/services/network => generated 88 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 12:00:29.525560 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae11e73-c601-4f1f-ad63-4468c1bdd535 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:29.552016 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becab7da-cb31-4b6c-b714-9af517b206fe None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:29.599624 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c154aabb-bfdf-4a05-953e-dafee8d379a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:29.600251 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c154aabb-bfdf-4a05-953e-dafee8d379a7 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:29.600532 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c154aabb-bfdf-4a05-953e-dafee8d379a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:29.600806 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c154aabb-bfdf-4a05-953e-dafee8d379a7 None admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:29.601462 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c154aabb-bfdf-4a05-953e-dafee8d379a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:29.601762 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c154aabb-bfdf-4a05-953e-dafee8d379a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:29.603161 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c154aabb-bfdf-4a05-953e-dafee8d379a7 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:29.628893 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 418/826] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 9 12:00:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:00:29.644188 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c154aabb-bfdf-4a05-953e-dafee8d379a7 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:29.725140 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae11e73-c601-4f1f-ad63-4468c1bdd535 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:29.731260 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:29.768375 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c154aabb-bfdf-4a05-953e-dafee8d379a7 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:29.798089 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab33acf-2f09-4b99-917a-d2bae7406a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:29.798701 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab33acf-2f09-4b99-917a-d2bae7406a02 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:29.799009 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab33acf-2f09-4b99-917a-d2bae7406a02 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:29.799262 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab33acf-2f09-4b99-917a-d2bae7406a02 None admin] PATH_INFO: `/v3/services` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:29.800443 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab33acf-2f09-4b99-917a-d2bae7406a02 None admin] RBAC: Adding query filter params (name=network) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:29.800902 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab33acf-2f09-4b99-917a-d2bae7406a02 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:29.802587 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab33acf-2f09-4b99-917a-d2bae7406a02 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:29.811763 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 409/827] 10.222.0.49 () {56 vars in 1112 bytes} [Fri Sep 9 12:00:29 2022] GET /identity/v3/services?name=network => generated 124 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:29.818281 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab33acf-2f09-4b99-917a-d2bae7406a02 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:29.842607 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee5d9a9-13d3-4cd1-9bb8-d914578583d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:29.843653 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee5d9a9-13d3-4cd1-9bb8-d914578583d2 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:29.844113 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee5d9a9-13d3-4cd1-9bb8-d914578583d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:29.844581 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee5d9a9-13d3-4cd1-9bb8-d914578583d2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:29.855958 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee5d9a9-13d3-4cd1-9bb8-d914578583d2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:29.857445 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee5d9a9-13d3-4cd1-9bb8-d914578583d2 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:29.882389 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-aee5d9a9-13d3-4cd1-9bb8-d914578583d2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:00:29.882389 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:29.882389 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:29.882389 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:29.882389 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:29.882389 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:29.882389 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:29.882389 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:29.882389 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:29.882389 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:29.882389 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:29.882389 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:29.882389 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 12:00:29.882389 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 12:00:29.882389 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 12:00:29.883384 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 12:00:29.883384 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:29.883384 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:29.883384 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:29.883384 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:29.883384 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:29.883384 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:29.883384 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:29.883384 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:29.883384 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:29.883384 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:29.883384 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:29.883384 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:29.883384 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:29.883384 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:29.883384 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:29.883384 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:29.884260 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 12:00:29.884260 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 12:00:29.884260 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 12:00:29.884260 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 12:00:29.884260 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 12:00:29.884260 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 12:00:29.884260 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:00:29.884260 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:00:29.891466 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 419/828] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 9 12:00:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:29.897699 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee5d9a9-13d3-4cd1-9bb8-d914578583d2 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:29.981884 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:30.061720 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab33acf-2f09-4b99-917a-d2bae7406a02 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:30.102189 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcae495-f2af-4df2-be8c-c7dd1f5df17e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:30.103129 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcae495-f2af-4df2-be8c-c7dd1f5df17e None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:30.103602 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcae495-f2af-4df2-be8c-c7dd1f5df17e None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:30.104058 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcae495-f2af-4df2-be8c-c7dd1f5df17e None admin] PATH_INFO: `/v3/services` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:30.104863 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcae495-f2af-4df2-be8c-c7dd1f5df17e None admin] RBAC: Adding query filter params (type=network) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:30.105756 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcae495-f2af-4df2-be8c-c7dd1f5df17e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:30.107358 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcae495-f2af-4df2-be8c-c7dd1f5df17e None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:30.113933 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 410/829] 10.222.0.49 () {56 vars in 1112 bytes} [Fri Sep 9 12:00:29 2022] GET /identity/v3/services?type=network => generated 350 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:00:30.124550 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcae495-f2af-4df2-be8c-c7dd1f5df17e None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:30.161923 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee5d9a9-13d3-4cd1-9bb8-d914578583d2 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:30.220537 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eae642c-0ab5-49fb-bbca-fa9695f7e88e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:30.221156 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eae642c-0ab5-49fb-bbca-fa9695f7e88e None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:30.221434 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eae642c-0ab5-49fb-bbca-fa9695f7e88e None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:30.221806 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eae642c-0ab5-49fb-bbca-fa9695f7e88e None admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:30.222694 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eae642c-0ab5-49fb-bbca-fa9695f7e88e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:30.223101 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eae642c-0ab5-49fb-bbca-fa9695f7e88e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:30.224529 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eae642c-0ab5-49fb-bbca-fa9695f7e88e None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:30.227279 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:30.245827 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 420/830] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Sep 9 12:00:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:00:30.252248 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eae642c-0ab5-49fb-bbca-fa9695f7e88e None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:30.299287 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:30.312136 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcae495-f2af-4df2-be8c-c7dd1f5df17e None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:30.348322 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef739cc-fba6-42b1-a03d-d9bc70ce3dc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:30.348999 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef739cc-fba6-42b1-a03d-d9bc70ce3dc9 None admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:30.349241 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef739cc-fba6-42b1-a03d-d9bc70ce3dc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:30.349516 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef739cc-fba6-42b1-a03d-d9bc70ce3dc9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:30.350175 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef739cc-fba6-42b1-a03d-d9bc70ce3dc9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:30.351787 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef739cc-fba6-42b1-a03d-d9bc70ce3dc9 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:30.373981 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 411/831] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:00:30 2022] POST /identity/v3/endpoints => generated 337 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:00:30.380079 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a072e683-b758-407b-8f2a-fe7b1b2a00ab None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:30.380324 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a072e683-b758-407b-8f2a-fe7b1b2a00ab None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:30.380600 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a072e683-b758-407b-8f2a-fe7b1b2a00ab None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:30.386362 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 412/832] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:00:30 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:00:30.388782 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694a1626-3db6-4300-a5a9-b97ba8341838 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:30.388782 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694a1626-3db6-4300-a5a9-b97ba8341838 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:30.388782 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694a1626-3db6-4300-a5a9-b97ba8341838 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:30.403168 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eae642c-0ab5-49fb-bbca-fa9695f7e88e None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:30.420276 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-694a1626-3db6-4300-a5a9-b97ba8341838 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:00:30.421498 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-694a1626-3db6-4300-a5a9-b97ba8341838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:30.468562 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2956adcb-5614-4fe3-8fd4-2151a1ba3c8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:30.469441 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2956adcb-5614-4fe3-8fd4-2151a1ba3c8c None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:30.469717 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2956adcb-5614-4fe3-8fd4-2151a1ba3c8c None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:30.469988 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2956adcb-5614-4fe3-8fd4-2151a1ba3c8c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:30.470720 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2956adcb-5614-4fe3-8fd4-2151a1ba3c8c None admin] RBAC: Adding query filter params (role.id=3f2ca33902a04b6dbc2ef7d06d7289d3, scope.project.id=fe863fcd75f1406ebbffc5016078f73e, user.id=70bac169d5cd43e1926f310344ac11fa) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:30.472416 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2956adcb-5614-4fe3-8fd4-2151a1ba3c8c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:30.473970 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2956adcb-5614-4fe3-8fd4-2151a1ba3c8c None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:30.490370 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 421/833] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Sep 9 12:00:30 2022] GET /identity/v3/role_assignments?user.id=70bac169d5cd43e1926f310344ac11fa&scope.project.id=fe863fcd75f1406ebbffc5016078f73e&role.id=3f2ca33902a04b6dbc2ef7d06d7289d3 => generated 608 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:30.528196 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 413/834] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:00:30 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 12:00:30.865486 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359326a5-e0b5-4328-a9d1-572638ce86a1 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:30.866007 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359326a5-e0b5-4328-a9d1-572638ce86a1 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:30.866007 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359326a5-e0b5-4328-a9d1-572638ce86a1 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:30.867590 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 422/835] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:00:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:00:30.873019 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e981ec-ec92-4f39-b168-c95179dafd0a None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:30.873308 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e981ec-ec92-4f39-b168-c95179dafd0a None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:30.873582 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e981ec-ec92-4f39-b168-c95179dafd0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:30.890321 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359326a5-e0b5-4328-a9d1-572638ce86a1 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:30.894472 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-e0e981ec-ec92-4f39-b168-c95179dafd0a None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:00:30.895679 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-e0e981ec-ec92-4f39-b168-c95179dafd0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:30.957009 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:30.971000 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 414/836] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:00:30 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 12:00:31.008142 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359326a5-e0b5-4328-a9d1-572638ce86a1 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:31.043185 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a5c388-c527-4293-8a86-cce6afdeda7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:31.044170 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a5c388-c527-4293-8a86-cce6afdeda7c None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:31.044657 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a5c388-c527-4293-8a86-cce6afdeda7c None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:31.045149 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a5c388-c527-4293-8a86-cce6afdeda7c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:31.056783 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a5c388-c527-4293-8a86-cce6afdeda7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:00:31.061952 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-37a5c388-c527-4293-8a86-cce6afdeda7c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 12:00:31.061952 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:31.061952 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:31.061952 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:31.061952 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:31.061952 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:31.061952 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:31.061952 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:31.061952 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:31.061952 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:31.061952 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:31.061952 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:31.061952 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:00:31.061952 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:00:31.061952 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:00:31.061952 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:00:31.062874 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:00:31.062874 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:00:31.062874 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:00:31.062874 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:00:31.062874 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:31.062874 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:31.062874 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:31.062874 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:31.062874 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:31.062874 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:31.062874 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:31.062874 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:31.062874 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:31.062874 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:31.062874 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:31.062874 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:31.063890 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:31.063890 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:31.063890 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:31.063890 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:31.063890 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:00:31.063890 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:00:31.063890 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:00:31.063890 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:00:31.063890 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:00:31.063890 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:00:31.063890 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 12:00:31.063890 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:00:31.067082 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 423/837] 10.222.0.49 () {56 vars in 1102 bytes} [Fri Sep 9 12:00:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 12:00:31.073167 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e981ec-ec92-4f39-b168-c95179dafd0a None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:31.087259 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:31.179912 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e981ec-ec92-4f39-b168-c95179dafd0a None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:31.215266 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181f62c4-1426-4756-aeb3-545f0a8b248b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:31.215880 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181f62c4-1426-4756-aeb3-545f0a8b248b None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:31.216155 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181f62c4-1426-4756-aeb3-545f0a8b248b None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:31.216425 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181f62c4-1426-4756-aeb3-545f0a8b248b None admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:31.217088 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181f62c4-1426-4756-aeb3-545f0a8b248b None admin] RBAC: Adding query filter params (name=service) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:31.217381 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181f62c4-1426-4756-aeb3-545f0a8b248b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:31.218745 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181f62c4-1426-4756-aeb3-545f0a8b248b None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:31.228064 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 415/838] 10.222.0.49 () {56 vars in 1103 bytes} [Fri Sep 9 12:00:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:31.233712 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a5c388-c527-4293-8a86-cce6afdeda7c None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:31.257767 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181f62c4-1426-4756-aeb3-545f0a8b248b None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:31.341371 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:31.362822 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a5c388-c527-4293-8a86-cce6afdeda7c None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:31.381419 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181f62c4-1426-4756-aeb3-545f0a8b248b None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:31.385352 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:31.406559 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34c961f-5e7a-4c3a-9d30-fe632acd9d64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:31.407375 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34c961f-5e7a-4c3a-9d30-fe632acd9d64 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:31.407746 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34c961f-5e7a-4c3a-9d30-fe632acd9d64 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:31.408110 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34c961f-5e7a-4c3a-9d30-fe632acd9d64 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:31.421380 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34c961f-5e7a-4c3a-9d30-fe632acd9d64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:31.423362 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34c961f-5e7a-4c3a-9d30-fe632acd9d64 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:31.427100 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abd73ad-4e54-4da3-9b5d-148130ce9e2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:31.427943 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abd73ad-4e54-4da3-9b5d-148130ce9e2e None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:31.428365 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abd73ad-4e54-4da3-9b5d-148130ce9e2e None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:31.428774 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abd73ad-4e54-4da3-9b5d-148130ce9e2e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:31.439838 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-a34c961f-5e7a-4c3a-9d30-fe632acd9d64 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:00:31.439838 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:31.439838 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:31.439838 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:31.439838 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:31.439838 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:31.439838 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:31.439838 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:31.439838 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:31.439838 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:31.439838 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:31.439838 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:31.439838 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:00:31.439838 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:00:31.439838 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:00:31.439838 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:00:31.440827 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:31.440827 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:31.440827 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:31.440827 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:31.440827 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:31.440827 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:31.440827 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:31.440827 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:31.440827 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:31.440827 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:31.440827 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:31.440827 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:31.440827 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:31.440827 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:31.440827 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:31.440827 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:31.440827 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:00:31.442177 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:00:31.442177 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:00:31.442177 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:00:31.442502 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abd73ad-4e54-4da3-9b5d-148130ce9e2e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:00:31.442577 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 424/839] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 12:00:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:31.442884 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eebde9b-25f1-4b62-89fc-cafb19d8868a None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:31.443181 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eebde9b-25f1-4b62-89fc-cafb19d8868a None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:31.443448 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eebde9b-25f1-4b62-89fc-cafb19d8868a None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:31.444865 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 425/840] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:00:31 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:00:31.446806 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eebde9b-25f1-4b62-89fc-cafb19d8868a None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:31.450345 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-8abd73ad-4e54-4da3-9b5d-148130ce9e2e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 12:00:31.450345 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:31.450345 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:31.450345 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:31.450345 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:31.450345 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:31.450345 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:31.450345 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:31.450345 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:31.450345 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:31.450345 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:31.450345 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:31.450345 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:00:31.450345 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:00:31.450345 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:00:31.450345 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:00:31.451313 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:00:31.451313 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:00:31.451313 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:00:31.451313 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:00:31.451313 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:31.451313 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:31.451313 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:31.451313 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:31.451313 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:31.451313 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:31.451313 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:31.451313 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:31.451313 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:31.451313 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:31.451313 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:31.451313 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:31.452157 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:31.452157 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:31.452157 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:31.452157 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:31.452157 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:00:31.452157 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:00:31.452157 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:00:31.452157 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:00:31.452157 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:00:31.452157 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:00:31.452157 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 12:00:31.452157 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:00:31.452157 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 416/841] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Sep 9 12:00:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 12:00:31.459012 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0372c8b9-b080-4051-91fc-665d0e725b35 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:31.459012 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0372c8b9-b080-4051-91fc-665d0e725b35 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:31.459012 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0372c8b9-b080-4051-91fc-665d0e725b35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:31.476135 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-0372c8b9-b080-4051-91fc-665d0e725b35 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:00:31.477329 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-0372c8b9-b080-4051-91fc-665d0e725b35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:31.548240 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 417/842] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:00:31 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 12:00:31.549446 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0372c8b9-b080-4051-91fc-665d0e725b35 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:31.568995 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eebde9b-25f1-4b62-89fc-cafb19d8868a None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:31.610699 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037675a6-bc88-44a2-8b11-52fb83f7750b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:31.611545 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037675a6-bc88-44a2-8b11-52fb83f7750b None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:31.611888 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037675a6-bc88-44a2-8b11-52fb83f7750b None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:31.612872 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037675a6-bc88-44a2-8b11-52fb83f7750b None admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:31.613768 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037675a6-bc88-44a2-8b11-52fb83f7750b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:31.614095 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037675a6-bc88-44a2-8b11-52fb83f7750b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:31.615568 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037675a6-bc88-44a2-8b11-52fb83f7750b None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:31.628301 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:31.639075 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 426/843] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 9 12:00:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:31.649425 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037675a6-bc88-44a2-8b11-52fb83f7750b None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:31.683394 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:31.690845 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0372c8b9-b080-4051-91fc-665d0e725b35 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:31.726449 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a618576-dec5-4704-9848-d17b5d41d3ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:31.727296 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a618576-dec5-4704-9848-d17b5d41d3ca None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:31.727691 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a618576-dec5-4704-9848-d17b5d41d3ca None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:31.728033 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a618576-dec5-4704-9848-d17b5d41d3ca None admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:31.728762 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a618576-dec5-4704-9848-d17b5d41d3ca None admin] RBAC: Adding query filter params (name=admin) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:31.729131 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a618576-dec5-4704-9848-d17b5d41d3ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:31.730610 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a618576-dec5-4704-9848-d17b5d41d3ca None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:31.739484 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 418/844] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 9 12:00:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:31.744966 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a618576-dec5-4704-9848-d17b5d41d3ca None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:31.771743 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037675a6-bc88-44a2-8b11-52fb83f7750b None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:31.824718 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f824f76-44f4-4189-9cfa-cfa002df1348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:31.825341 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f824f76-44f4-4189-9cfa-cfa002df1348 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:31.825613 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f824f76-44f4-4189-9cfa-cfa002df1348 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:31.825874 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f824f76-44f4-4189-9cfa-cfa002df1348 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:31.836078 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f824f76-44f4-4189-9cfa-cfa002df1348 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:31.837520 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f824f76-44f4-4189-9cfa-cfa002df1348 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:31.849611 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-6f824f76-44f4-4189-9cfa-cfa002df1348 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 12:00:31.849611 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:31.849611 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:31.849611 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:31.849611 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:31.849611 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:31.849611 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:31.849611 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:31.849611 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:31.849611 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:31.849611 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:31.849611 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:31.849611 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 12:00:31.849611 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 12:00:31.849611 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 12:00:31.849611 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 12:00:31.850576 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:31.850576 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:31.850576 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 12:00:31.850576 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:00:31.850576 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 12:00:31.850576 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:00:31.850576 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:31.850576 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:31.850576 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:31.850576 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:31.850576 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:31.850576 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:31.850576 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:31.850576 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:31.850576 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:31.850576 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:31.850576 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:31.850576 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:31.851581 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:31.851581 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:31.851581 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 12:00:31.851581 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 12:00:31.851581 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 12:00:31.851581 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 12:00:31.851581 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 12:00:31.851581 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 12:00:31.851581 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 12:00:31.851581 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:00:31.851581 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 427/845] 10.222.0.49 () {56 vars in 1117 bytes} [Fri Sep 9 12:00:31 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:31.858809 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f824f76-44f4-4189-9cfa-cfa002df1348 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:31.870608 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a618576-dec5-4704-9848-d17b5d41d3ca None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:31.893033 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:31.908685 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71af95e-7925-4b5d-bdd4-9821508861bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:31.909923 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71af95e-7925-4b5d-bdd4-9821508861bc None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:31.910337 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71af95e-7925-4b5d-bdd4-9821508861bc None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:31.910498 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71af95e-7925-4b5d-bdd4-9821508861bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:31.921207 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71af95e-7925-4b5d-bdd4-9821508861bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:31.922636 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71af95e-7925-4b5d-bdd4-9821508861bc None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:31.939234 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-d71af95e-7925-4b5d-bdd4-9821508861bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:00:31.939234 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:31.939234 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:31.939234 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:31.939234 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:31.939234 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:31.939234 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:31.939234 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:31.939234 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:31.939234 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:31.939234 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:31.939234 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:31.939234 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:00:31.939234 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:00:31.939234 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:00:31.939234 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:00:31.941078 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:31.941078 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:31.941078 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:31.941078 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:31.941078 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:31.941078 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:31.941078 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:31.941078 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:31.941078 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:31.941078 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:31.941078 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:31.941078 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:31.941078 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:31.941078 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:31.941078 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:31.941078 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:31.941078 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:00:31.941990 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:00:31.941990 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:00:31.941990 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:00:31.942749 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 419/846] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 12:00:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:31.943662 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71af95e-7925-4b5d-bdd4-9821508861bc None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:31.956182 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:31.969609 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f824f76-44f4-4189-9cfa-cfa002df1348 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:32.006347 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f661cbf2-c62e-40fb-b015-ea8f7dcab001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:32.007011 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f661cbf2-c62e-40fb-b015-ea8f7dcab001 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:32.007312 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f661cbf2-c62e-40fb-b015-ea8f7dcab001 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:32.007565 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f661cbf2-c62e-40fb-b015-ea8f7dcab001 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:32.008145 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f661cbf2-c62e-40fb-b015-ea8f7dcab001 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:32.009494 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f661cbf2-c62e-40fb-b015-ea8f7dcab001 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:32.018931 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-f661cbf2-c62e-40fb-b015-ea8f7dcab001 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 12:00:32.018931 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:32.018931 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 12:00:32.018931 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 12:00:32.018931 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 12:00:32.018931 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 12:00:32.018931 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 12:00:32.018931 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 12:00:32.018931 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 12:00:32.018931 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:00:32.018931 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 12:00:32.018931 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:00:32.018931 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:32.018931 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:32.018931 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:32.018931 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:32.018931 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:32.020393 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:32.020393 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:32.020393 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:32.020393 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:32.020393 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:32.020393 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:32.020393 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 12:00:32.020393 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 12:00:32.020393 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 12:00:32.020393 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 12:00:32.020393 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:32.020393 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:32.020393 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:32.020393 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:32.020393 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:32.020393 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:32.021222 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:32.021222 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:32.021222 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:32.021222 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:32.021222 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:32.021222 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:32.021222 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:32.021222 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:32.021222 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:32.021222 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:32.021222 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 12:00:32.021222 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 12:00:32.021222 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 12:00:32.021222 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:00:32.021222 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 428/847] 10.222.0.49 () {56 vars in 1114 bytes} [Fri Sep 9 12:00:31 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:32.022026 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f661cbf2-c62e-40fb-b015-ea8f7dcab001 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:32.052961 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71af95e-7925-4b5d-bdd4-9821508861bc None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:32.091166 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d3d889-3f11-4736-8385-e825abc4b9ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:32.091849 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d3d889-3f11-4736-8385-e825abc4b9ed None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:32.092139 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d3d889-3f11-4736-8385-e825abc4b9ed None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:32.092298 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d3d889-3f11-4736-8385-e825abc4b9ed None admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:32.093013 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d3d889-3f11-4736-8385-e825abc4b9ed None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:32.093312 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d3d889-3f11-4736-8385-e825abc4b9ed None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:32.094727 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d3d889-3f11-4736-8385-e825abc4b9ed None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:32.115017 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 420/848] 10.222.0.49 () {56 vars in 1149 bytes} [Fri Sep 9 12:00:31 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:32.115017 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d3d889-3f11-4736-8385-e825abc4b9ed None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:32.136565 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f661cbf2-c62e-40fb-b015-ea8f7dcab001 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:32.152276 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:32.171039 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b8bab6-30bc-4264-aae6-afb6980f2440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:32.171665 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b8bab6-30bc-4264-aae6-afb6980f2440 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:32.171943 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b8bab6-30bc-4264-aae6-afb6980f2440 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:32.172213 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b8bab6-30bc-4264-aae6-afb6980f2440 None admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:32.172842 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b8bab6-30bc-4264-aae6-afb6980f2440 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:32.173140 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b8bab6-30bc-4264-aae6-afb6980f2440 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:32.174485 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b8bab6-30bc-4264-aae6-afb6980f2440 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:32.190839 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 429/849] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 9 12:00:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:32.190839 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b8bab6-30bc-4264-aae6-afb6980f2440 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:32.201917 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:32.234609 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d3d889-3f11-4736-8385-e825abc4b9ed None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:32.269159 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3af88a-4bd1-4678-9d36-b7e15858fb98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:32.269837 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3af88a-4bd1-4678-9d36-b7e15858fb98 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:32.270111 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3af88a-4bd1-4678-9d36-b7e15858fb98 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:32.270371 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3af88a-4bd1-4678-9d36-b7e15858fb98 None admin] PATH_INFO: `/v3/services` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:32.271094 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3af88a-4bd1-4678-9d36-b7e15858fb98 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:32.271394 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3af88a-4bd1-4678-9d36-b7e15858fb98 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:32.272733 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3af88a-4bd1-4678-9d36-b7e15858fb98 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:32.277330 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 421/850] 10.222.0.49 () {56 vars in 1114 bytes} [Fri Sep 9 12:00:32 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:32.277894 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3af88a-4bd1-4678-9d36-b7e15858fb98 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:32.295931 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b8bab6-30bc-4264-aae6-afb6980f2440 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:32.332278 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e2c162-96fb-4ed7-bfac-f247c2b7967e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:32.332880 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e2c162-96fb-4ed7-bfac-f247c2b7967e None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:32.333156 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e2c162-96fb-4ed7-bfac-f247c2b7967e None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:32.333428 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e2c162-96fb-4ed7-bfac-f247c2b7967e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:32.374069 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:32.382373 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3af88a-4bd1-4678-9d36-b7e15858fb98 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:32.405428 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e2c162-96fb-4ed7-bfac-f247c2b7967e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:32.406877 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e2c162-96fb-4ed7-bfac-f247c2b7967e None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:32.417299 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09be2ae3-fc8b-4686-a748-32925a7f1c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:32.417924 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09be2ae3-fc8b-4686-a748-32925a7f1c05 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:32.418194 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09be2ae3-fc8b-4686-a748-32925a7f1c05 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:32.418462 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09be2ae3-fc8b-4686-a748-32925a7f1c05 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:32.421551 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:32.426416 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09be2ae3-fc8b-4686-a748-32925a7f1c05 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:32.428360 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09be2ae3-fc8b-4686-a748-32925a7f1c05 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:32.443930 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-09be2ae3-fc8b-4686-a748-32925a7f1c05 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 09 12:00:32.443930 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:32.443930 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:32.443930 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:32.443930 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:32.443930 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:32.443930 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:32.443930 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:32.443930 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:32.443930 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:32.443930 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:32.443930 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:32.443930 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 12:00:32.443930 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 12:00:32.443930 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 12:00:32.443930 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 12:00:32.444854 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:32.444854 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:32.444854 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 12:00:32.444854 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:00:32.444854 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 12:00:32.444854 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:00:32.444854 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:32.444854 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:32.444854 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:32.444854 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:32.444854 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:32.444854 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:32.444854 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:32.444854 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:32.444854 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:32.444854 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:32.444854 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:32.444854 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:32.445780 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:32.445780 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:32.445780 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 12:00:32.445780 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 12:00:32.445780 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 12:00:32.445780 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 12:00:32.445780 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 12:00:32.445780 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 12:00:32.445780 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 09 12:00:32.445780 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:00:32.451487 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 422/851] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Sep 9 12:00:32 2022] GET /identity/v3/users/swift => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:32.451487 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09be2ae3-fc8b-4686-a748-32925a7f1c05 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:32.513089 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 430/852] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 9 12:00:32 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 12:00:32.513960 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e2c162-96fb-4ed7-bfac-f247c2b7967e None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:32.564057 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09be2ae3-fc8b-4686-a748-32925a7f1c05 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:32.602049 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cf23d7-d50d-4255-beb7-903336b5f37a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:32.603080 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cf23d7-d50d-4255-beb7-903336b5f37a None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:32.603671 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cf23d7-d50d-4255-beb7-903336b5f37a None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:32.604094 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cf23d7-d50d-4255-beb7-903336b5f37a None admin] PATH_INFO: `/v3/services` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:32.604716 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cf23d7-d50d-4255-beb7-903336b5f37a None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:32.605062 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cf23d7-d50d-4255-beb7-903336b5f37a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:32.606559 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cf23d7-d50d-4255-beb7-903336b5f37a None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:32.622944 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 423/853] 10.222.0.49 () {56 vars in 1114 bytes} [Fri Sep 9 12:00:32 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:32.622944 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cf23d7-d50d-4255-beb7-903336b5f37a None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:32.633581 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e2c162-96fb-4ed7-bfac-f247c2b7967e None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:32.644194 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:32.655078 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:32.681966 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c6f39d-91f9-478f-bc0d-799750c09945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:32.682584 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c6f39d-91f9-478f-bc0d-799750c09945 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:32.682855 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c6f39d-91f9-478f-bc0d-799750c09945 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:32.683155 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c6f39d-91f9-478f-bc0d-799750c09945 None admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:32.683903 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c6f39d-91f9-478f-bc0d-799750c09945 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:32.684203 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c6f39d-91f9-478f-bc0d-799750c09945 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:32.685662 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c6f39d-91f9-478f-bc0d-799750c09945 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:32.710988 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 431/854] 10.222.0.49 () {56 vars in 1135 bytes} [Fri Sep 9 12:00:32 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:32.712421 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c6f39d-91f9-478f-bc0d-799750c09945 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:32.758380 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cf23d7-d50d-4255-beb7-903336b5f37a None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:32.798208 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da727353-7fcb-4f1a-bd01-d9584e0db8b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:32.799017 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da727353-7fcb-4f1a-bd01-d9584e0db8b4 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:32.799178 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da727353-7fcb-4f1a-bd01-d9584e0db8b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:32.799555 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da727353-7fcb-4f1a-bd01-d9584e0db8b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:32.812889 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da727353-7fcb-4f1a-bd01-d9584e0db8b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:32.816505 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da727353-7fcb-4f1a-bd01-d9584e0db8b4 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:32.834121 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-da727353-7fcb-4f1a-bd01-d9584e0db8b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:00:32.834121 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:32.834121 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:32.834121 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:32.834121 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:32.834121 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:32.834121 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:32.834121 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:32.834121 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:32.834121 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:32.834121 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:32.834121 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:32.834121 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:00:32.834121 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:00:32.834121 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:00:32.834121 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:00:32.835167 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:32.835167 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:32.835167 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:32.835167 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:32.835167 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:32.835167 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:32.835167 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:32.835167 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:32.835167 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:32.835167 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:32.835167 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:32.835167 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:32.835167 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:32.835167 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:32.835167 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:32.835167 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:32.835167 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:00:32.836060 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:00:32.836060 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:00:32.836060 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:00:32.840296 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 424/855] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 12:00:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:32.841324 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da727353-7fcb-4f1a-bd01-d9584e0db8b4 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:32.842938 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c6f39d-91f9-478f-bc0d-799750c09945 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:32.887357 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49425984-f3d7-40fd-ae51-a9ee6a225467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:32.887992 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49425984-f3d7-40fd-ae51-a9ee6a225467 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:32.888268 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49425984-f3d7-40fd-ae51-a9ee6a225467 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:32.888534 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49425984-f3d7-40fd-ae51-a9ee6a225467 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:32.934242 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:32.935908 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:32.970322 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49425984-f3d7-40fd-ae51-a9ee6a225467 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:32.972631 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49425984-f3d7-40fd-ae51-a9ee6a225467 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:32.980776 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da727353-7fcb-4f1a-bd01-d9584e0db8b4 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:33.017249 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8321a3c1-a781-4e96-beb2-15bebfbacb6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:33.018188 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8321a3c1-a781-4e96-beb2-15bebfbacb6e None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:33.019049 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8321a3c1-a781-4e96-beb2-15bebfbacb6e None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:33.019355 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8321a3c1-a781-4e96-beb2-15bebfbacb6e None admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:33.019989 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8321a3c1-a781-4e96-beb2-15bebfbacb6e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:33.020285 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8321a3c1-a781-4e96-beb2-15bebfbacb6e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:33.021644 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8321a3c1-a781-4e96-beb2-15bebfbacb6e None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:33.034781 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 425/856] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 9 12:00:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:33.036050 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506c9884-2502-4368-b5b2-80925f432f15 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:33.036322 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506c9884-2502-4368-b5b2-80925f432f15 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:33.036583 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506c9884-2502-4368-b5b2-80925f432f15 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:33.041708 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 426/857] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:00:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:00:33.041708 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506c9884-2502-4368-b5b2-80925f432f15 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:33.087398 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 432/858] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 9 12:00:32 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 09 12:00:33.088666 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20215631-65ac-4da5-8c1f-7a92f32e219e None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:33.089021 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20215631-65ac-4da5-8c1f-7a92f32e219e None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:33.089404 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20215631-65ac-4da5-8c1f-7a92f32e219e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:33.114568 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-20215631-65ac-4da5-8c1f-7a92f32e219e None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:00:33.115784 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-20215631-65ac-4da5-8c1f-7a92f32e219e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:33.153032 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506c9884-2502-4368-b5b2-80925f432f15 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:33.185179 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:33.192982 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:33.193538 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99705cfd-5706-4041-92ac-aaa140b93aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:33.194185 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99705cfd-5706-4041-92ac-aaa140b93aac None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:33.194447 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99705cfd-5706-4041-92ac-aaa140b93aac None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:33.194712 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99705cfd-5706-4041-92ac-aaa140b93aac None admin] PATH_INFO: `/v3/projects/service` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:33.203067 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 433/859] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:00:33 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 12:00:33.203067 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20215631-65ac-4da5-8c1f-7a92f32e219e None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:33.204237 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99705cfd-5706-4041-92ac-aaa140b93aac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:33.205679 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99705cfd-5706-4041-92ac-aaa140b93aac None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:33.220516 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-99705cfd-5706-4041-92ac-aaa140b93aac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:00:33.220516 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:33.220516 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:33.220516 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:33.220516 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:33.220516 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:33.220516 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:33.220516 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:33.220516 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:33.220516 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:33.220516 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:33.220516 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:33.220516 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 12:00:33.220516 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 12:00:33.220516 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 12:00:33.222228 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 12:00:33.222228 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:33.222228 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:33.222228 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:33.222228 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:33.222228 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:33.222228 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:33.222228 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:33.222228 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:33.222228 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:33.222228 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:33.222228 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:33.222228 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:33.222228 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:33.222228 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:33.222228 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:33.222228 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:33.223389 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 12:00:33.223389 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 12:00:33.223389 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 12:00:33.223389 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 12:00:33.223389 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 12:00:33.223389 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 12:00:33.223389 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:00:33.223389 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:00:33.223389 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 427/860] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 9 12:00:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:33.228593 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99705cfd-5706-4041-92ac-aaa140b93aac None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:33.326243 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20215631-65ac-4da5-8c1f-7a92f32e219e None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:33.338140 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99705cfd-5706-4041-92ac-aaa140b93aac None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:33.368411 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eec0d01-adc7-43d3-86af-4d2ccdbfc35a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:33.369345 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eec0d01-adc7-43d3-86af-4d2ccdbfc35a None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:33.369771 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eec0d01-adc7-43d3-86af-4d2ccdbfc35a None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:33.370224 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eec0d01-adc7-43d3-86af-4d2ccdbfc35a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:33.376991 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4082b2-9443-4130-9de1-bfef52d5b275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:33.377742 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4082b2-9443-4130-9de1-bfef52d5b275 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:33.378618 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4082b2-9443-4130-9de1-bfef52d5b275 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:33.379024 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4082b2-9443-4130-9de1-bfef52d5b275 None admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:33.380223 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4082b2-9443-4130-9de1-bfef52d5b275 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:33.380223 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4082b2-9443-4130-9de1-bfef52d5b275 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:33.382088 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4082b2-9443-4130-9de1-bfef52d5b275 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:33.384281 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eec0d01-adc7-43d3-86af-4d2ccdbfc35a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:33.385996 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eec0d01-adc7-43d3-86af-4d2ccdbfc35a None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:33.396296 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 428/861] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Sep 9 12:00:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:00:33.402805 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-4eec0d01-adc7-43d3-86af-4d2ccdbfc35a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:00:33.402805 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:33.402805 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:33.402805 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:33.402805 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:33.402805 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:33.402805 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:33.402805 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:33.402805 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:33.402805 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:33.402805 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:33.402805 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:33.402805 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:00:33.402805 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:00:33.402805 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:00:33.402805 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:00:33.403777 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:33.403777 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:33.403777 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:33.403777 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:33.403777 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:33.403777 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:33.403777 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:33.403777 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:33.403777 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:33.403777 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:33.403777 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:33.403777 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:33.403777 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:33.403777 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:33.403777 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:33.403777 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:33.403777 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:00:33.404712 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:00:33.404712 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:00:33.404712 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:00:33.404712 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 434/862] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 12:00:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:33.405272 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4082b2-9443-4130-9de1-bfef52d5b275 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:33.410725 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eec0d01-adc7-43d3-86af-4d2ccdbfc35a None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:33.463876 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:33.472760 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:33.525872 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4082b2-9443-4130-9de1-bfef52d5b275 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:33.526615 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eec0d01-adc7-43d3-86af-4d2ccdbfc35a None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:33.564235 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d66da2-0e76-4262-b2e4-9442fb679b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:33.565121 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d66da2-0e76-4262-b2e4-9442fb679b1c None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:33.566172 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d66da2-0e76-4262-b2e4-9442fb679b1c None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:33.566424 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d66da2-0e76-4262-b2e4-9442fb679b1c None admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:33.567099 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d66da2-0e76-4262-b2e4-9442fb679b1c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:33.567404 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d66da2-0e76-4262-b2e4-9442fb679b1c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:33.568810 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d66da2-0e76-4262-b2e4-9442fb679b1c None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:33.569290 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3903c77e-5f94-44a3-879d-26ee73e515df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:33.570144 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3903c77e-5f94-44a3-879d-26ee73e515df None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:33.571005 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3903c77e-5f94-44a3-879d-26ee73e515df None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:33.571438 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3903c77e-5f94-44a3-879d-26ee73e515df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:33.572884 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3903c77e-5f94-44a3-879d-26ee73e515df None admin] RBAC: Adding query filter params (role.id=8abeff2cb87d4b7fa72b27b028005877, scope.project.id=fe863fcd75f1406ebbffc5016078f73e, user.id=e368b4e668c54e25bff80a8e8a8c3588) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:33.573353 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3903c77e-5f94-44a3-879d-26ee73e515df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:33.575011 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3903c77e-5f94-44a3-879d-26ee73e515df None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:33.582821 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 435/863] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 9 12:00:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:33.583718 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d66da2-0e76-4262-b2e4-9442fb679b1c None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:33.584279 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 429/864] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Sep 9 12:00:33 2022] GET /identity/v3/role_assignments?user.id=e368b4e668c54e25bff80a8e8a8c3588&scope.project.id=fe863fcd75f1406ebbffc5016078f73e&role.id=8abeff2cb87d4b7fa72b27b028005877 => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:33.588793 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3903c77e-5f94-44a3-879d-26ee73e515df None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:33.705044 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d66da2-0e76-4262-b2e4-9442fb679b1c None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:33.712380 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3903c77e-5f94-44a3-879d-26ee73e515df None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:33.722567 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:33.731943 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:33.754702 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b686e53e-ed7d-4fac-a995-224fa9a91401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:33.755803 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b686e53e-ed7d-4fac-a995-224fa9a91401 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:33.756285 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b686e53e-ed7d-4fac-a995-224fa9a91401 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:33.756662 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b686e53e-ed7d-4fac-a995-224fa9a91401 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:33.757497 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b686e53e-ed7d-4fac-a995-224fa9a91401 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:33.759723 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b686e53e-ed7d-4fac-a995-224fa9a91401 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:33.760134 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d41846-7c91-4716-92d8-fa150234fd61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:33.760933 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d41846-7c91-4716-92d8-fa150234fd61 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:33.761357 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d41846-7c91-4716-92d8-fa150234fd61 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:33.761631 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d41846-7c91-4716-92d8-fa150234fd61 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:33.770789 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d41846-7c91-4716-92d8-fa150234fd61 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:33.772407 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d41846-7c91-4716-92d8-fa150234fd61 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:33.772757 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-b686e53e-ed7d-4fac-a995-224fa9a91401 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 12:00:33.772757 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:33.772757 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 12:00:33.772757 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 12:00:33.772757 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 12:00:33.772757 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 12:00:33.772757 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 12:00:33.772757 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 12:00:33.772757 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 12:00:33.772757 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:00:33.772757 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 12:00:33.772757 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:00:33.772757 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:33.772757 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:33.772757 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:33.772757 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:33.773836 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:33.773836 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:33.773836 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:33.773836 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:33.773836 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:33.773836 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:33.773836 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:33.773836 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 12:00:33.773836 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 12:00:33.773836 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 12:00:33.773836 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 12:00:33.773836 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:33.773836 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:33.773836 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:33.773836 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:33.773836 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:33.773836 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:33.774882 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:33.774882 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:33.774882 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:33.774882 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:33.774882 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:33.774882 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:33.774882 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:33.774882 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:33.774882 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:33.774882 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:33.774882 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 12:00:33.774882 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 12:00:33.774882 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 12:00:33.774882 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:00:33.776930 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 436/865] 10.222.0.49 () {56 vars in 1132 bytes} [Fri Sep 9 12:00:33 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:33.782736 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b686e53e-ed7d-4fac-a995-224fa9a91401 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:33.786213 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-03d41846-7c91-4716-92d8-fa150234fd61 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:00:33.786213 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:33.786213 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:33.786213 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:33.786213 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:33.786213 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:33.786213 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:33.786213 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:33.786213 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:33.786213 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:33.786213 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:33.786213 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:33.786213 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 12:00:33.786213 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 12:00:33.786213 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 12:00:33.787379 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 12:00:33.787379 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:33.787379 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:33.787379 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:33.787379 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:33.787379 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:33.787379 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:33.787379 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:33.787379 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:33.787379 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:33.787379 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:33.787379 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:33.787379 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:33.787379 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:33.787379 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:33.787379 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:33.787379 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:33.788704 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 12:00:33.788704 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 12:00:33.788704 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 12:00:33.788704 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 12:00:33.788704 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 12:00:33.788704 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 12:00:33.788704 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:00:33.788704 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:00:33.788704 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 430/866] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 9 12:00:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:33.795388 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d41846-7c91-4716-92d8-fa150234fd61 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:33.931095 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d41846-7c91-4716-92d8-fa150234fd61 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:33.933252 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b686e53e-ed7d-4fac-a995-224fa9a91401 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:33.969761 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5398decb-9a7f-461b-9e9c-c31b0de6c612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:33.970748 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5398decb-9a7f-461b-9e9c-c31b0de6c612 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:33.972076 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5398decb-9a7f-461b-9e9c-c31b0de6c612 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:33.972495 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5398decb-9a7f-461b-9e9c-c31b0de6c612 None admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:33.973821 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5398decb-9a7f-461b-9e9c-c31b0de6c612 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:33.974609 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5398decb-9a7f-461b-9e9c-c31b0de6c612 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:33.976288 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5398decb-9a7f-461b-9e9c-c31b0de6c612 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:33.978560 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2876de-b500-4187-9fa1-dcb897854a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:33.979272 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2876de-b500-4187-9fa1-dcb897854a0e None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:33.980067 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2876de-b500-4187-9fa1-dcb897854a0e None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:33.980428 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2876de-b500-4187-9fa1-dcb897854a0e None admin] PATH_INFO: `/v3/services` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:33.981136 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2876de-b500-4187-9fa1-dcb897854a0e None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:33.981510 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2876de-b500-4187-9fa1-dcb897854a0e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:33.982960 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2876de-b500-4187-9fa1-dcb897854a0e None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:33.992873 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 437/867] 10.222.0.49 () {56 vars in 1126 bytes} [Fri Sep 9 12:00:33 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:33.994135 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 431/868] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Sep 9 12:00:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:00:33.995350 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2876de-b500-4187-9fa1-dcb897854a0e None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:34.004613 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5398decb-9a7f-461b-9e9c-c31b0de6c612 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:34.048582 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:34.053092 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:34.118868 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2876de-b500-4187-9fa1-dcb897854a0e None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:34.125773 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5398decb-9a7f-461b-9e9c-c31b0de6c612 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:34.156646 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c39067-699b-4ffb-bec7-3920e6b5555b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:34.157557 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c39067-699b-4ffb-bec7-3920e6b5555b None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:34.157974 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c39067-699b-4ffb-bec7-3920e6b5555b None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:34.158423 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c39067-699b-4ffb-bec7-3920e6b5555b None admin] PATH_INFO: `/v3/services` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:34.159493 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c39067-699b-4ffb-bec7-3920e6b5555b None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:34.160068 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c39067-699b-4ffb-bec7-3920e6b5555b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:34.161687 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c39067-699b-4ffb-bec7-3920e6b5555b None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:34.162302 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9452030c-3114-434f-bcff-7573766d3788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:34.163021 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9452030c-3114-434f-bcff-7573766d3788 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:34.163367 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9452030c-3114-434f-bcff-7573766d3788 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:34.163706 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9452030c-3114-434f-bcff-7573766d3788 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:34.164490 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9452030c-3114-434f-bcff-7573766d3788 None admin] RBAC: Adding query filter params (role.id=3f2ca33902a04b6dbc2ef7d06d7289d3, scope.project.id=fe863fcd75f1406ebbffc5016078f73e, user.id=92e652ba827849b5a3fc1213d04a1b76) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:34.164879 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9452030c-3114-434f-bcff-7573766d3788 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:34.166364 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9452030c-3114-434f-bcff-7573766d3788 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:34.175639 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 438/870] 10.222.0.49 () {56 vars in 1126 bytes} [Fri Sep 9 12:00:33 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:34.178861 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 432/870] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Sep 9 12:00:34 2022] GET /identity/v3/role_assignments?user.id=92e652ba827849b5a3fc1213d04a1b76&scope.project.id=fe863fcd75f1406ebbffc5016078f73e&role.id=3f2ca33902a04b6dbc2ef7d06d7289d3 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:34.906814 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d49192-6b6d-4dd6-8bf4-f359187c381a None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:34.907030 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d49192-6b6d-4dd6-8bf4-f359187c381a None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:34.907300 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d49192-6b6d-4dd6-8bf4-f359187c381a None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:34.909113 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 439/871] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:00:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:00:34.914599 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425e42e9-c129-4666-bf52-2295ad16cd0d None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:34.914882 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425e42e9-c129-4666-bf52-2295ad16cd0d None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:34.915185 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425e42e9-c129-4666-bf52-2295ad16cd0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:34.934471 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-425e42e9-c129-4666-bf52-2295ad16cd0d None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:00:34.935685 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-425e42e9-c129-4666-bf52-2295ad16cd0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:35.008836 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 433/872] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:00:34 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 12:00:35.285436 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d49192-6b6d-4dd6-8bf4-f359187c381a None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:35.392027 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d49192-6b6d-4dd6-8bf4-f359187c381a None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:35.395122 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:35.430371 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1140e3a1-a3f1-4e15-9c31-0ed3bff4b246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:35.431363 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1140e3a1-a3f1-4e15-9c31-0ed3bff4b246 None admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:35.431857 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1140e3a1-a3f1-4e15-9c31-0ed3bff4b246 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:35.432318 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1140e3a1-a3f1-4e15-9c31-0ed3bff4b246 None admin] PATH_INFO: `/v3/services` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:35.433164 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1140e3a1-a3f1-4e15-9c31-0ed3bff4b246 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:35.434730 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1140e3a1-a3f1-4e15-9c31-0ed3bff4b246 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:35.452166 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 440/873] 10.222.0.49 () {60 vars in 1141 bytes} [Fri Sep 9 12:00:35 2022] POST /identity/v3/services => generated 251 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:00:35.912197 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1017891d-e40c-4211-971b-6cade5525915 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:35.912686 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1017891d-e40c-4211-971b-6cade5525915 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:35.912686 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1017891d-e40c-4211-971b-6cade5525915 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:35.914217 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 434/874] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:00:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:00:35.919691 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fc47fe-c016-4837-8e4f-82da19f9724c None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:35.919978 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fc47fe-c016-4837-8e4f-82da19f9724c None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:35.920253 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fc47fe-c016-4837-8e4f-82da19f9724c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:35.939148 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-a5fc47fe-c016-4837-8e4f-82da19f9724c None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:00:35.941209 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-a5fc47fe-c016-4837-8e4f-82da19f9724c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:36.014154 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 441/875] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:00:35 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 12:00:36.286436 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1017891d-e40c-4211-971b-6cade5525915 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:36.314916 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:36.404437 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1017891d-e40c-4211-971b-6cade5525915 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:36.441213 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1271b8c3-1a65-412e-952b-f02c68c6537d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:36.441842 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1271b8c3-1a65-412e-952b-f02c68c6537d None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:36.442117 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1271b8c3-1a65-412e-952b-f02c68c6537d None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:36.442382 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1271b8c3-1a65-412e-952b-f02c68c6537d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:36.453482 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1271b8c3-1a65-412e-952b-f02c68c6537d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:00:36.459185 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-1271b8c3-1a65-412e-952b-f02c68c6537d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 12:00:36.459185 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:36.459185 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:36.459185 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:36.459185 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:36.459185 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:36.459185 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:36.459185 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:36.459185 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:36.459185 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:36.459185 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:36.459185 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:36.459185 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:00:36.459185 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:00:36.459185 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:00:36.459185 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:00:36.460096 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:00:36.460096 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:00:36.460096 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:00:36.460096 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:00:36.460096 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:36.460096 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:36.460096 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:36.460096 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:36.460096 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:36.460096 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:36.460096 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:36.460096 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:36.460096 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:36.460096 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:36.460096 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:36.460096 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:36.460940 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:36.460940 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:36.460940 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:36.460940 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:36.460940 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:00:36.460940 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:00:36.460940 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:00:36.460940 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:00:36.460940 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:00:36.460940 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:00:36.460940 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 12:00:36.460940 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:00:36.460940 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 435/876] 10.222.0.49 () {56 vars in 1120 bytes} [Fri Sep 9 12:00:36 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:36.466580 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fc47fe-c016-4837-8e4f-82da19f9724c None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:36.483877 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea6f3d9-5022-49a3-934f-c9eab44298ea None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:36.484166 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea6f3d9-5022-49a3-934f-c9eab44298ea None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:36.484441 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea6f3d9-5022-49a3-934f-c9eab44298ea None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:36.486456 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 436/877] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:00:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 12:00:36.488261 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5c1021-6e8a-4f56-bc63-f21b899fe883 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:36.488682 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5c1021-6e8a-4f56-bc63-f21b899fe883 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:36.489028 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5c1021-6e8a-4f56-bc63-f21b899fe883 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:36.490572 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 437/878] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:00:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:00:36.493557 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21bf6f5-6e90-4f7b-afca-5b842ec7bb19 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:36.494082 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21bf6f5-6e90-4f7b-afca-5b842ec7bb19 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:36.494190 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21bf6f5-6e90-4f7b-afca-5b842ec7bb19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:36.512910 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-f21bf6f5-6e90-4f7b-afca-5b842ec7bb19 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:00:36.514098 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-f21bf6f5-6e90-4f7b-afca-5b842ec7bb19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:36.565962 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:36.578079 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fc47fe-c016-4837-8e4f-82da19f9724c None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:36.587016 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 438/879] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:00:36 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 12:00:36.588223 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cfd989-7cc2-419d-9dd8-07ed85510116 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:36.588512 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cfd989-7cc2-419d-9dd8-07ed85510116 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:36.588801 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cfd989-7cc2-419d-9dd8-07ed85510116 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:36.608062 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-81cfd989-7cc2-419d-9dd8-07ed85510116 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:00:36.609279 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-81cfd989-7cc2-419d-9dd8-07ed85510116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:36.621864 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8354f00d-bda7-46eb-8eeb-197a3a5d1fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:36.622811 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8354f00d-bda7-46eb-8eeb-197a3a5d1fbd None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:36.623277 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:36.623736 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8354f00d-bda7-46eb-8eeb-197a3a5d1fbd None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:36.624175 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8354f00d-bda7-46eb-8eeb-197a3a5d1fbd None admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:36.625008 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8354f00d-bda7-46eb-8eeb-197a3a5d1fbd None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:36.625501 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8354f00d-bda7-46eb-8eeb-197a3a5d1fbd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:36.627048 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8354f00d-bda7-46eb-8eeb-197a3a5d1fbd None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:36.638052 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 442/880] 10.222.0.49 () {56 vars in 1115 bytes} [Fri Sep 9 12:00:36 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:36.643969 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8354f00d-bda7-46eb-8eeb-197a3a5d1fbd None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:36.710946 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 439/881] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:00:36 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 12:00:36.755723 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8354f00d-bda7-46eb-8eeb-197a3a5d1fbd None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:36.790551 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608a5494-4e2c-4920-97fb-baf39a0842db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:36.791216 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608a5494-4e2c-4920-97fb-baf39a0842db None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:36.791496 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608a5494-4e2c-4920-97fb-baf39a0842db None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:36.791769 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608a5494-4e2c-4920-97fb-baf39a0842db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:36.803245 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608a5494-4e2c-4920-97fb-baf39a0842db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:36.804745 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608a5494-4e2c-4920-97fb-baf39a0842db None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:36.824678 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-608a5494-4e2c-4920-97fb-baf39a0842db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:00:36.824678 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:36.824678 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:36.824678 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:36.824678 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:36.824678 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:36.824678 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:36.824678 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:36.824678 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:36.824678 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:36.824678 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:36.824678 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:36.824678 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:00:36.824678 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:00:36.824678 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:00:36.824678 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:00:36.825628 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:36.825628 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:36.825628 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:36.825628 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:36.825628 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:36.825628 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:36.825628 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:36.825628 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:36.825628 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:36.825628 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:36.825628 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:36.825628 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:36.825628 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:36.825628 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:36.825628 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:36.825628 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:36.825628 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:00:36.826508 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:00:36.826508 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:00:36.826508 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:00:36.826508 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 443/882] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 12:00:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 12:00:36.831978 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cfd989-7cc2-419d-9dd8-07ed85510116 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:36.880922 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608a5494-4e2c-4920-97fb-baf39a0842db None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:36.884824 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:36.942751 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cfd989-7cc2-419d-9dd8-07ed85510116 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:36.977835 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178a0579-5efb-4d95-9720-db7fe61faddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:36.978447 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178a0579-5efb-4d95-9720-db7fe61faddc None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:36.978728 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178a0579-5efb-4d95-9720-db7fe61faddc None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:36.979139 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178a0579-5efb-4d95-9720-db7fe61faddc None admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:36.979782 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178a0579-5efb-4d95-9720-db7fe61faddc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:36.980085 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178a0579-5efb-4d95-9720-db7fe61faddc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:36.981459 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178a0579-5efb-4d95-9720-db7fe61faddc None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:36.985951 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608a5494-4e2c-4920-97fb-baf39a0842db None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:36.993364 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 440/883] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 9 12:00:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:36.995640 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178a0579-5efb-4d95-9720-db7fe61faddc None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:37.008010 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:37.023419 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c582498f-ed22-4a73-8942-5edd36ffa6a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:37.024262 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c582498f-ed22-4a73-8942-5edd36ffa6a2 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:37.024741 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c582498f-ed22-4a73-8942-5edd36ffa6a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:37.025153 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c582498f-ed22-4a73-8942-5edd36ffa6a2 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:37.025937 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c582498f-ed22-4a73-8942-5edd36ffa6a2 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:37.027467 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c582498f-ed22-4a73-8942-5edd36ffa6a2 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:37.037369 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-c582498f-ed22-4a73-8942-5edd36ffa6a2 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 12:00:37.037369 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:37.037369 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 12:00:37.037369 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 12:00:37.037369 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 12:00:37.037369 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 12:00:37.037369 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 12:00:37.037369 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 12:00:37.037369 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 12:00:37.037369 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:00:37.037369 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 12:00:37.037369 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:00:37.037369 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:37.037369 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:37.037369 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:37.037369 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:37.037369 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:37.038624 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:37.038624 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:37.038624 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:37.038624 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:37.038624 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:37.038624 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:37.038624 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 12:00:37.038624 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 12:00:37.038624 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 12:00:37.038624 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 12:00:37.038624 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:37.038624 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:37.038624 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:37.038624 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:37.038624 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:37.038624 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:37.039751 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:37.039751 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:37.039751 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:37.039751 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:37.039751 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:37.039751 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:37.039751 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:37.039751 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:37.039751 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:37.039751 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:37.039751 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 12:00:37.039751 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 12:00:37.039751 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 12:00:37.039751 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:00:37.042124 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 444/884] 10.222.0.49 () {56 vars in 1126 bytes} [Fri Sep 9 12:00:36 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 12:00:37.042891 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c582498f-ed22-4a73-8942-5edd36ffa6a2 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:37.109525 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178a0579-5efb-4d95-9720-db7fe61faddc None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:37.118947 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:37.147969 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512a88f0-489f-4515-b1aa-1a7b3ad21dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:37.148575 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512a88f0-489f-4515-b1aa-1a7b3ad21dba None admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:37.148852 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512a88f0-489f-4515-b1aa-1a7b3ad21dba None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:37.149125 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512a88f0-489f-4515-b1aa-1a7b3ad21dba None admin] PATH_INFO: `/v3/services` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:37.149771 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512a88f0-489f-4515-b1aa-1a7b3ad21dba None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:37.151150 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512a88f0-489f-4515-b1aa-1a7b3ad21dba None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:37.156984 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c582498f-ed22-4a73-8942-5edd36ffa6a2 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:37.166678 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 441/885] 10.222.0.49 () {60 vars in 1141 bytes} [Fri Sep 9 12:00:36 2022] POST /identity/v3/services => generated 269 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:00:37.167717 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512a88f0-489f-4515-b1aa-1a7b3ad21dba None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:37.215098 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3928febf-6523-44c0-b04b-59e3d609a517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:37.215937 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3928febf-6523-44c0-b04b-59e3d609a517 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:37.216276 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3928febf-6523-44c0-b04b-59e3d609a517 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:37.216613 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3928febf-6523-44c0-b04b-59e3d609a517 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:37.231601 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3928febf-6523-44c0-b04b-59e3d609a517 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:37.233482 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3928febf-6523-44c0-b04b-59e3d609a517 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:37.249463 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-3928febf-6523-44c0-b04b-59e3d609a517 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 12:00:37.249463 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:37.249463 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:37.249463 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:37.249463 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:37.249463 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:37.249463 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:37.249463 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:37.249463 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:37.249463 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:37.249463 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:37.249463 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:37.249463 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 12:00:37.249463 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 12:00:37.249463 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 12:00:37.249463 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 12:00:37.250729 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:37.250729 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:37.250729 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 12:00:37.250729 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:00:37.250729 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 12:00:37.250729 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:00:37.250729 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:37.250729 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:37.250729 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:37.250729 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:37.250729 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:37.250729 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:37.250729 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:37.250729 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:37.250729 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:37.250729 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:37.250729 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:37.250729 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:37.251976 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:37.251976 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:37.251976 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 12:00:37.251976 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 12:00:37.251976 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 12:00:37.251976 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 12:00:37.251976 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 12:00:37.251976 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 12:00:37.251976 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 12:00:37.251976 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:00:37.255192 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 445/886] 10.222.0.49 () {56 vars in 1117 bytes} [Fri Sep 9 12:00:37 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:37.260294 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:37.267958 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3928febf-6523-44c0-b04b-59e3d609a517 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:37.302184 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512a88f0-489f-4515-b1aa-1a7b3ad21dba None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:37.342091 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d69955-7b90-4802-9c04-597234491696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:37.342709 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d69955-7b90-4802-9c04-597234491696 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:37.343012 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d69955-7b90-4802-9c04-597234491696 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:37.343282 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d69955-7b90-4802-9c04-597234491696 None admin] PATH_INFO: `/v3/services` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:37.344026 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d69955-7b90-4802-9c04-597234491696 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:37.344329 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d69955-7b90-4802-9c04-597234491696 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:37.345903 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d69955-7b90-4802-9c04-597234491696 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:37.352284 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 442/887] 10.222.0.49 () {56 vars in 1122 bytes} [Fri Sep 9 12:00:37 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:37.357988 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d69955-7b90-4802-9c04-597234491696 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:37.382491 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3928febf-6523-44c0-b04b-59e3d609a517 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:37.413089 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:37.425400 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46e4920-c429-49bc-aa5f-c416cf481ffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:37.425992 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46e4920-c429-49bc-aa5f-c416cf481ffd None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:37.426261 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46e4920-c429-49bc-aa5f-c416cf481ffd None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:37.426532 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46e4920-c429-49bc-aa5f-c416cf481ffd None admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:37.427337 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46e4920-c429-49bc-aa5f-c416cf481ffd None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:37.427648 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46e4920-c429-49bc-aa5f-c416cf481ffd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:37.429086 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46e4920-c429-49bc-aa5f-c416cf481ffd None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:37.448632 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 446/888] 10.222.0.49 () {56 vars in 1149 bytes} [Fri Sep 9 12:00:37 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:37.455270 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46e4920-c429-49bc-aa5f-c416cf481ffd None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:37.500845 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d69955-7b90-4802-9c04-597234491696 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:37.540947 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:37.549053 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b698153-c47e-4f61-b366-b2e62e4bf308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:37.550365 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b698153-c47e-4f61-b366-b2e62e4bf308 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:37.550887 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b698153-c47e-4f61-b366-b2e62e4bf308 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:37.551829 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b698153-c47e-4f61-b366-b2e62e4bf308 None admin] PATH_INFO: `/v3/services` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:37.552934 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b698153-c47e-4f61-b366-b2e62e4bf308 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:37.553433 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b698153-c47e-4f61-b366-b2e62e4bf308 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:37.555596 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b698153-c47e-4f61-b366-b2e62e4bf308 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:37.562654 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 443/889] 10.222.0.49 () {56 vars in 1122 bytes} [Fri Sep 9 12:00:37 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:37.576361 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46e4920-c429-49bc-aa5f-c416cf481ffd None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:37.618838 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c868db-d03b-4d51-b27c-14a020cb8dd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:37.619904 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c868db-d03b-4d51-b27c-14a020cb8dd2 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:37.620386 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c868db-d03b-4d51-b27c-14a020cb8dd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:37.620857 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c868db-d03b-4d51-b27c-14a020cb8dd2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:37.681961 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:37.703997 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c868db-d03b-4d51-b27c-14a020cb8dd2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:37.705479 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c868db-d03b-4d51-b27c-14a020cb8dd2 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:37.757048 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a139833-86f7-4a74-a57a-b3a82d85d298 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:37.757679 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a139833-86f7-4a74-a57a-b3a82d85d298 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:37.758018 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a139833-86f7-4a74-a57a-b3a82d85d298 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:37.759477 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 444/890] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:00:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:00:37.765146 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3c825c-83f0-4ef8-bfa2-1f51c512c606 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:37.765439 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3c825c-83f0-4ef8-bfa2-1f51c512c606 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:37.765717 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3c825c-83f0-4ef8-bfa2-1f51c512c606 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:37.790564 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-ea3c825c-83f0-4ef8-bfa2-1f51c512c606 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:00:37.791846 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-ea3c825c-83f0-4ef8-bfa2-1f51c512c606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:37.819883 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 447/891] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 9 12:00:37 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 09 12:00:37.825727 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c868db-d03b-4d51-b27c-14a020cb8dd2 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:37.878935 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 445/892] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:00:37 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 12:00:37.915263 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:37.934016 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c868db-d03b-4d51-b27c-14a020cb8dd2 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:37.969749 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88baf403-c0d9-4ecb-9388-c45a2afd6a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:37.970390 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88baf403-c0d9-4ecb-9388-c45a2afd6a54 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:37.970665 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88baf403-c0d9-4ecb-9388-c45a2afd6a54 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:37.970957 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88baf403-c0d9-4ecb-9388-c45a2afd6a54 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:37.982032 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88baf403-c0d9-4ecb-9388-c45a2afd6a54 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:37.983517 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88baf403-c0d9-4ecb-9388-c45a2afd6a54 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:37.999479 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-88baf403-c0d9-4ecb-9388-c45a2afd6a54 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:00:37.999479 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:37.999479 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:37.999479 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:37.999479 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:37.999479 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:37.999479 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:37.999479 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:37.999479 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:37.999479 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:37.999479 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:37.999479 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:37.999479 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:00:37.999479 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:00:37.999479 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:00:37.999479 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:00:38.000467 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:38.000467 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:38.000467 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:38.000467 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:38.000467 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:38.000467 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:38.000467 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:38.000467 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:38.000467 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:38.000467 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:38.000467 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:38.000467 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:38.000467 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:38.000467 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:38.000467 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:38.000467 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:38.000467 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:00:38.001370 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:00:38.001370 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:00:38.001370 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:00:38.001370 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 448/893] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 12:00:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 12:00:38.008041 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3c825c-83f0-4ef8-bfa2-1f51c512c606 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:38.115926 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3c825c-83f0-4ef8-bfa2-1f51c512c606 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:38.147760 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:38.159441 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458b7798-340c-4652-81bb-7896be8c1adb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:38.160075 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458b7798-340c-4652-81bb-7896be8c1adb None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:38.160408 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458b7798-340c-4652-81bb-7896be8c1adb None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:38.160684 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458b7798-340c-4652-81bb-7896be8c1adb None admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:38.161346 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458b7798-340c-4652-81bb-7896be8c1adb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:38.161647 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458b7798-340c-4652-81bb-7896be8c1adb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:38.163099 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458b7798-340c-4652-81bb-7896be8c1adb None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:38.172392 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88baf403-c0d9-4ecb-9388-c45a2afd6a54 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:38.176903 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 446/894] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 9 12:00:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:38.183880 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458b7798-340c-4652-81bb-7896be8c1adb None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:38.287952 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88baf403-c0d9-4ecb-9388-c45a2afd6a54 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:38.293371 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458b7798-340c-4652-81bb-7896be8c1adb None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:38.327215 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373847b9-ba0e-4c76-b850-765b544e43a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:38.327974 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373847b9-ba0e-4c76-b850-765b544e43a2 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:38.328385 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373847b9-ba0e-4c76-b850-765b544e43a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:38.328674 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373847b9-ba0e-4c76-b850-765b544e43a2 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:38.329260 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373847b9-ba0e-4c76-b850-765b544e43a2 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:38.330689 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373847b9-ba0e-4c76-b850-765b544e43a2 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:38.332623 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:38.334126 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e596832-8758-409d-acac-8b6aa0c8260d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:38.335036 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e596832-8758-409d-acac-8b6aa0c8260d None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:38.335564 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e596832-8758-409d-acac-8b6aa0c8260d None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:38.336010 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e596832-8758-409d-acac-8b6aa0c8260d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:38.340038 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-373847b9-ba0e-4c76-b850-765b544e43a2 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 12:00:38.340038 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:38.340038 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 12:00:38.340038 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 12:00:38.340038 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 12:00:38.340038 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 12:00:38.340038 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 12:00:38.340038 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 12:00:38.340038 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 12:00:38.340038 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:00:38.340038 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 12:00:38.340038 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:00:38.340038 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:38.340038 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:38.340038 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:38.340038 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:38.340038 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:38.341018 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:38.341018 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:38.341018 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:38.341018 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:38.341018 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:38.341018 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:38.341018 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 12:00:38.341018 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 12:00:38.341018 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 12:00:38.341018 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 12:00:38.341018 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:38.341018 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:38.341018 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:38.341018 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:38.341018 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:38.341018 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:38.341860 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:38.341860 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:38.341860 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:38.341860 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:38.341860 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:38.341860 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:38.341860 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:38.341860 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:38.341860 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:38.341860 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:38.341860 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 12:00:38.341860 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 12:00:38.341860 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 12:00:38.341860 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:00:38.346126 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 449/895] 10.222.0.49 () {56 vars in 1114 bytes} [Fri Sep 9 12:00:38 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 12:00:38.347545 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373847b9-ba0e-4c76-b850-765b544e43a2 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:38.349467 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e596832-8758-409d-acac-8b6aa0c8260d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:38.350947 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e596832-8758-409d-acac-8b6aa0c8260d None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:38.365734 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-9e596832-8758-409d-acac-8b6aa0c8260d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:00:38.365734 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:38.365734 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:38.365734 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:38.365734 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:38.365734 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:38.365734 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:38.365734 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:38.365734 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:38.365734 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:38.365734 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:38.365734 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:38.365734 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 12:00:38.365734 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 12:00:38.365734 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 12:00:38.366703 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 12:00:38.366703 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:38.366703 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:38.366703 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:38.366703 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:38.366703 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:38.366703 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:38.366703 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:38.366703 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:38.366703 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:38.366703 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:38.366703 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:38.366703 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:38.366703 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:38.366703 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:38.366703 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:38.366703 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:38.367624 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 12:00:38.367624 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 12:00:38.367624 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 12:00:38.367624 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 12:00:38.367624 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 12:00:38.367624 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 12:00:38.367624 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:00:38.367624 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:00:38.371897 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 447/896] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 9 12:00:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:38.373159 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e596832-8758-409d-acac-8b6aa0c8260d None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:38.434878 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:38.478151 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373847b9-ba0e-4c76-b850-765b544e43a2 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:38.509911 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e596832-8758-409d-acac-8b6aa0c8260d None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:38.520405 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732a703b-32e2-4023-9251-ac5091d4d08b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:38.521019 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732a703b-32e2-4023-9251-ac5091d4d08b None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:38.521399 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732a703b-32e2-4023-9251-ac5091d4d08b None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:38.521733 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732a703b-32e2-4023-9251-ac5091d4d08b None admin] PATH_INFO: `/v3/services` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:38.522375 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732a703b-32e2-4023-9251-ac5091d4d08b None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:38.522773 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732a703b-32e2-4023-9251-ac5091d4d08b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:38.524232 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732a703b-32e2-4023-9251-ac5091d4d08b None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:38.530100 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 450/897] 10.222.0.49 () {56 vars in 1114 bytes} [Fri Sep 9 12:00:38 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:38.536505 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732a703b-32e2-4023-9251-ac5091d4d08b None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:38.559205 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f78f08-146c-4a4b-8226-06ba0bea0183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:38.559815 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f78f08-146c-4a4b-8226-06ba0bea0183 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:38.560094 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f78f08-146c-4a4b-8226-06ba0bea0183 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:38.560368 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f78f08-146c-4a4b-8226-06ba0bea0183 None admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:38.561536 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f78f08-146c-4a4b-8226-06ba0bea0183 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:38.561839 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f78f08-146c-4a4b-8226-06ba0bea0183 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:38.563600 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f78f08-146c-4a4b-8226-06ba0bea0183 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:38.583069 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 448/898] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Sep 9 12:00:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:38.588397 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f78f08-146c-4a4b-8226-06ba0bea0183 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:38.610504 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:38.667032 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732a703b-32e2-4023-9251-ac5091d4d08b None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:38.709086 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cca30b2-5526-4b06-8bb4-6fff84816392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:38.710075 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cca30b2-5526-4b06-8bb4-6fff84816392 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:38.710651 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cca30b2-5526-4b06-8bb4-6fff84816392 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:38.711126 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cca30b2-5526-4b06-8bb4-6fff84816392 None admin] PATH_INFO: `/v3/services` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:38.711927 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cca30b2-5526-4b06-8bb4-6fff84816392 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:38.712405 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cca30b2-5526-4b06-8bb4-6fff84816392 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:38.714259 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cca30b2-5526-4b06-8bb4-6fff84816392 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:38.719952 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 451/899] 10.222.0.49 () {56 vars in 1114 bytes} [Fri Sep 9 12:00:38 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:00:38.723813 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f78f08-146c-4a4b-8226-06ba0bea0183 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:38.725481 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cca30b2-5526-4b06-8bb4-6fff84816392 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:38.732361 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:38.764027 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f0ec93-679c-4c46-a411-f1e069833e84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:38.764917 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f0ec93-679c-4c46-a411-f1e069833e84 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:38.765404 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f0ec93-679c-4c46-a411-f1e069833e84 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:38.765849 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f0ec93-679c-4c46-a411-f1e069833e84 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:38.766838 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f0ec93-679c-4c46-a411-f1e069833e84 None admin] RBAC: Adding query filter params (role.id=db537712f45d4e6980592d3ddefb312a, scope.project.id=fe863fcd75f1406ebbffc5016078f73e, user.id=e368b4e668c54e25bff80a8e8a8c3588) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:38.767345 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f0ec93-679c-4c46-a411-f1e069833e84 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:38.769064 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f0ec93-679c-4c46-a411-f1e069833e84 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:38.775442 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 449/900] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Sep 9 12:00:38 2022] GET /identity/v3/role_assignments?user.id=e368b4e668c54e25bff80a8e8a8c3588&scope.project.id=fe863fcd75f1406ebbffc5016078f73e&role.id=db537712f45d4e6980592d3ddefb312a => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:38.852661 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cca30b2-5526-4b06-8bb4-6fff84816392 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:38.883726 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:38.895677 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0a2a15-f531-4f52-b040-c1d5a7b30f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:38.896746 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0a2a15-f531-4f52-b040-c1d5a7b30f0a None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:38.897262 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0a2a15-f531-4f52-b040-c1d5a7b30f0a None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:38.897732 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0a2a15-f531-4f52-b040-c1d5a7b30f0a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:38.898654 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0a2a15-f531-4f52-b040-c1d5a7b30f0a None admin] RBAC: Adding query filter params (interface=public, service_id=608d22f72e964fcdb245358a5eb5eeb7, region_id=RegionOne) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:38.899193 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0a2a15-f531-4f52-b040-c1d5a7b30f0a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:38.900799 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0a2a15-f531-4f52-b040-c1d5a7b30f0a None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:38.907461 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 452/901] 10.222.0.49 () {56 vars in 1251 bytes} [Fri Sep 9 12:00:38 2022] GET /identity/v3/endpoints?service_id=608d22f72e964fcdb245358a5eb5eeb7&interface=public®ion_id=RegionOne => generated 194 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:38.913926 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f0ec93-679c-4c46-a411-f1e069833e84 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:39.022551 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f0ec93-679c-4c46-a411-f1e069833e84 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:39.057280 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f03ea2-0e84-49a3-8ed2-3dd106e61791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:39.057897 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f03ea2-0e84-49a3-8ed2-3dd106e61791 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:39.058158 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f03ea2-0e84-49a3-8ed2-3dd106e61791 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:39.058429 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f03ea2-0e84-49a3-8ed2-3dd106e61791 None admin] PATH_INFO: `/v3/services` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:39.059065 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f03ea2-0e84-49a3-8ed2-3dd106e61791 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:39.060437 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f03ea2-0e84-49a3-8ed2-3dd106e61791 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:39.067360 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 450/902] 10.222.0.49 () {56 vars in 1087 bytes} [Fri Sep 9 12:00:38 2022] GET /identity/v3/services => generated 1268 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:00:39.478267 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09f1b1b-cce3-4c00-a8ce-ea0cc3424883 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:39.478452 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09f1b1b-cce3-4c00-a8ce-ea0cc3424883 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:39.478718 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09f1b1b-cce3-4c00-a8ce-ea0cc3424883 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:39.480295 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 453/903] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:00:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:00:39.485688 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dd8822-20f5-4b53-9e6f-31d73c175830 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:39.485975 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dd8822-20f5-4b53-9e6f-31d73c175830 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:39.486248 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dd8822-20f5-4b53-9e6f-31d73c175830 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:39.506747 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-61dd8822-20f5-4b53-9e6f-31d73c175830 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:00:39.507990 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-61dd8822-20f5-4b53-9e6f-31d73c175830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:39.536885 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:39.581709 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 451/904] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:00:39 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 12:00:39.851515 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced3100c-205b-4dc4-b49b-fa9ddf6ace73 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:39.852071 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced3100c-205b-4dc4-b49b-fa9ddf6ace73 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:39.852071 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced3100c-205b-4dc4-b49b-fa9ddf6ace73 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:39.853602 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 454/905] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:00:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:00:39.856695 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dd8822-20f5-4b53-9e6f-31d73c175830 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:39.860769 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca87e3af-c3ea-49a7-9c86-d9bad54f4cb0 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:39.861155 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca87e3af-c3ea-49a7-9c86-d9bad54f4cb0 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:39.861524 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca87e3af-c3ea-49a7-9c86-d9bad54f4cb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:39.880278 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-ca87e3af-c3ea-49a7-9c86-d9bad54f4cb0 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:00:39.881487 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-ca87e3af-c3ea-49a7-9c86-d9bad54f4cb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:39.962315 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 455/906] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:00:39 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 12:00:39.975995 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dd8822-20f5-4b53-9e6f-31d73c175830 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:40.013714 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f04f3f-5bfb-4b7c-87a5-e034d6af1688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:40.014364 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f04f3f-5bfb-4b7c-87a5-e034d6af1688 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:40.014640 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f04f3f-5bfb-4b7c-87a5-e034d6af1688 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:40.014923 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f04f3f-5bfb-4b7c-87a5-e034d6af1688 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:40.015523 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f04f3f-5bfb-4b7c-87a5-e034d6af1688 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:40.016887 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f04f3f-5bfb-4b7c-87a5-e034d6af1688 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:40.026614 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-b4f04f3f-5bfb-4b7c-87a5-e034d6af1688 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 12:00:40.026614 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:40.026614 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 12:00:40.026614 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 12:00:40.026614 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 12:00:40.026614 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 12:00:40.026614 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 12:00:40.026614 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 12:00:40.026614 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 12:00:40.026614 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:00:40.026614 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 12:00:40.026614 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:00:40.026614 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:40.026614 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:40.026614 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:40.026614 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:40.028171 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:40.028171 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:40.028171 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:40.028171 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:40.028171 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:40.028171 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:40.028171 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:40.028171 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 12:00:40.028171 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 12:00:40.028171 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 12:00:40.028171 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 12:00:40.028171 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:40.028171 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:40.028171 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:40.028171 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:40.028171 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:40.028171 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:40.029298 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:40.029298 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:40.029298 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:40.029298 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:40.029298 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:40.029298 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:40.029298 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:40.029298 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:40.029298 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:40.029298 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:40.029298 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 12:00:40.029298 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 12:00:40.029298 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 12:00:40.029298 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:00:40.029298 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 452/907] 10.222.0.49 () {56 vars in 1132 bytes} [Fri Sep 9 12:00:39 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:40.033537 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca87e3af-c3ea-49a7-9c86-d9bad54f4cb0 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:40.138550 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca87e3af-c3ea-49a7-9c86-d9bad54f4cb0 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:40.159290 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:40.175010 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeb1fd6-e3ba-479a-b69f-ed82c965689c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:40.175885 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeb1fd6-e3ba-479a-b69f-ed82c965689c None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:40.176343 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeb1fd6-e3ba-479a-b69f-ed82c965689c None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:40.176800 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeb1fd6-e3ba-479a-b69f-ed82c965689c None admin] PATH_INFO: `/v3/services` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:40.177621 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aeb1fd6-e3ba-479a-b69f-ed82c965689c None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:40.178110 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aeb1fd6-e3ba-479a-b69f-ed82c965689c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:40.179746 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aeb1fd6-e3ba-479a-b69f-ed82c965689c None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:40.188776 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 456/908] 10.222.0.49 () {56 vars in 1126 bytes} [Fri Sep 9 12:00:40 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:40.195586 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f04f3f-5bfb-4b7c-87a5-e034d6af1688 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:40.226944 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:40.250819 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeb1fd6-e3ba-479a-b69f-ed82c965689c None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:40.312090 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f04f3f-5bfb-4b7c-87a5-e034d6af1688 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:40.360180 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f045b7d-997a-4448-9e87-cdc3c622345a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:40.361302 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f045b7d-997a-4448-9e87-cdc3c622345a None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:40.362264 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f045b7d-997a-4448-9e87-cdc3c622345a None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:40.362688 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f045b7d-997a-4448-9e87-cdc3c622345a None admin] PATH_INFO: `/v3/services` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:40.363647 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f045b7d-997a-4448-9e87-cdc3c622345a None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:40.364690 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f045b7d-997a-4448-9e87-cdc3c622345a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:40.366768 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f045b7d-997a-4448-9e87-cdc3c622345a None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:40.369954 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeb1fd6-e3ba-479a-b69f-ed82c965689c None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:40.377313 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 453/909] 10.222.0.49 () {56 vars in 1126 bytes} [Fri Sep 9 12:00:40 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:40.384634 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f045b7d-997a-4448-9e87-cdc3c622345a None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:40.414049 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3010b084-2cd6-4fec-b1ac-abf2e3c6c2a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:40.415130 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3010b084-2cd6-4fec-b1ac-abf2e3c6c2a1 None admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:40.415682 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3010b084-2cd6-4fec-b1ac-abf2e3c6c2a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:40.416153 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3010b084-2cd6-4fec-b1ac-abf2e3c6c2a1 None admin] PATH_INFO: `/v3/services` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:40.417002 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3010b084-2cd6-4fec-b1ac-abf2e3c6c2a1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:40.419426 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3010b084-2cd6-4fec-b1ac-abf2e3c6c2a1 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:40.443066 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 457/910] 10.222.0.49 () {60 vars in 1141 bytes} [Fri Sep 9 12:00:40 2022] POST /identity/v3/services => generated 241 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:00:40.489589 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:40.500660 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f045b7d-997a-4448-9e87-cdc3c622345a None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:40.551636 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105027a9-7017-4b0c-8967-5d1d2a0b6498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:40.552254 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105027a9-7017-4b0c-8967-5d1d2a0b6498 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:40.552531 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105027a9-7017-4b0c-8967-5d1d2a0b6498 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:40.552794 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105027a9-7017-4b0c-8967-5d1d2a0b6498 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:40.553485 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105027a9-7017-4b0c-8967-5d1d2a0b6498 None admin] RBAC: Adding query filter params (interface=public, service_id=13197247a71743089ceeba2f54f98b00, region_id=RegionOne) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:40.553787 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105027a9-7017-4b0c-8967-5d1d2a0b6498 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:40.555225 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105027a9-7017-4b0c-8967-5d1d2a0b6498 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:40.561596 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 454/911] 10.222.0.49 () {56 vars in 1251 bytes} [Fri Sep 9 12:00:40 2022] GET /identity/v3/endpoints?service_id=13197247a71743089ceeba2f54f98b00&interface=public®ion_id=RegionOne => generated 194 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:00:40.568445 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3010b084-2cd6-4fec-b1ac-abf2e3c6c2a1 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:40.623068 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:40.702785 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3010b084-2cd6-4fec-b1ac-abf2e3c6c2a1 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:40.739542 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffc7f33-b346-4f07-8a92-dee843cb960c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:40.740515 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffc7f33-b346-4f07-8a92-dee843cb960c None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:40.741024 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffc7f33-b346-4f07-8a92-dee843cb960c None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:40.741502 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffc7f33-b346-4f07-8a92-dee843cb960c None admin] PATH_INFO: `/v3/services` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:40.742395 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffc7f33-b346-4f07-8a92-dee843cb960c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:40.744036 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffc7f33-b346-4f07-8a92-dee843cb960c None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:40.750756 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 458/912] 10.222.0.49 () {56 vars in 1087 bytes} [Fri Sep 9 12:00:40 2022] GET /identity/v3/services => generated 1497 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:00:41.108090 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592c1504-aced-457a-bcff-017f3e24287b None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:41.108604 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592c1504-aced-457a-bcff-017f3e24287b None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:41.108604 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592c1504-aced-457a-bcff-017f3e24287b None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:41.110185 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 455/913] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:00:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:00:41.117074 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5c5d70-ac15-489f-9c16-f596f2dca5cc None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:41.117522 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5c5d70-ac15-489f-9c16-f596f2dca5cc None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:41.117619 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5c5d70-ac15-489f-9c16-f596f2dca5cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:41.136055 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-9d5c5d70-ac15-489f-9c16-f596f2dca5cc None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:00:41.137586 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-9d5c5d70-ac15-489f-9c16-f596f2dca5cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:41.209810 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 459/914] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:00:41 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 12:00:41.346834 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445fc58b-fcca-4894-b921-7e4ca7bbc1a8 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:41.347527 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445fc58b-fcca-4894-b921-7e4ca7bbc1a8 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:41.348014 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445fc58b-fcca-4894-b921-7e4ca7bbc1a8 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:41.349807 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 456/915] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:00:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:00:41.355368 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984d4470-cafd-4bd6-811a-ebe753455016 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:41.355919 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984d4470-cafd-4bd6-811a-ebe753455016 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:41.356414 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984d4470-cafd-4bd6-811a-ebe753455016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:41.375887 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-984d4470-cafd-4bd6-811a-ebe753455016 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:00:41.377979 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-984d4470-cafd-4bd6-811a-ebe753455016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:41.408918 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:41.453143 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 460/916] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:00:41 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 12:00:41.485408 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445fc58b-fcca-4894-b921-7e4ca7bbc1a8 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:41.596599 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445fc58b-fcca-4894-b921-7e4ca7bbc1a8 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:41.637011 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a60711-fa33-4f8a-8096-8aa97888bba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:41.637568 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a60711-fa33-4f8a-8096-8aa97888bba7 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:41.637846 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a60711-fa33-4f8a-8096-8aa97888bba7 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:41.638117 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a60711-fa33-4f8a-8096-8aa97888bba7 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:41.655244 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a60711-fa33-4f8a-8096-8aa97888bba7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:00:41.660187 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-93a60711-fa33-4f8a-8096-8aa97888bba7 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 12:00:41.660187 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:41.660187 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:41.660187 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:41.660187 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:41.660187 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:41.660187 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:41.660187 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:41.660187 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:41.660187 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:41.660187 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:41.660187 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:41.660187 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:00:41.660187 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:00:41.660187 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:00:41.660187 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:00:41.661112 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:00:41.661112 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:00:41.661112 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:00:41.661112 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:00:41.661112 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:41.661112 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:41.661112 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:41.661112 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:41.661112 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:41.661112 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:41.661112 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:41.661112 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:41.661112 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:41.661112 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:41.661112 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:41.661112 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:41.661953 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:41.661953 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:41.661953 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:41.661953 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:41.661953 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:00:41.661953 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:00:41.661953 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:00:41.661953 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:00:41.661953 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:00:41.661953 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:00:41.661953 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 12:00:41.661953 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:00:41.661953 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 457/917] 10.222.0.49 () {56 vars in 1120 bytes} [Fri Sep 9 12:00:41 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:41.668956 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984d4470-cafd-4bd6-811a-ebe753455016 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:41.743347 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a60711-fa33-4f8a-8096-8aa97888bba7 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:41.759601 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:41.781927 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984d4470-cafd-4bd6-811a-ebe753455016 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:41.820434 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb07ff11-0bf3-40df-b38e-a75488dc6803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:41.821048 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb07ff11-0bf3-40df-b38e-a75488dc6803 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:41.821326 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb07ff11-0bf3-40df-b38e-a75488dc6803 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:41.821596 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb07ff11-0bf3-40df-b38e-a75488dc6803 None admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:41.822263 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb07ff11-0bf3-40df-b38e-a75488dc6803 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:41.822566 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb07ff11-0bf3-40df-b38e-a75488dc6803 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:41.823964 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb07ff11-0bf3-40df-b38e-a75488dc6803 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:41.837728 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 461/918] 10.222.0.49 () {56 vars in 1115 bytes} [Fri Sep 9 12:00:41 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:41.838753 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb07ff11-0bf3-40df-b38e-a75488dc6803 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:41.864144 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a60711-fa33-4f8a-8096-8aa97888bba7 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:41.874746 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:41.916842 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b2bab6-a5c3-4568-9da9-353a62b7dd3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:41.918313 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b2bab6-a5c3-4568-9da9-353a62b7dd3c None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:41.918745 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b2bab6-a5c3-4568-9da9-353a62b7dd3c None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:41.919452 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b2bab6-a5c3-4568-9da9-353a62b7dd3c None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:41.920240 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b2bab6-a5c3-4568-9da9-353a62b7dd3c None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:41.922628 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b2bab6-a5c3-4568-9da9-353a62b7dd3c None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:41.933958 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-36b2bab6-a5c3-4568-9da9-353a62b7dd3c None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 12:00:41.933958 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:41.933958 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 12:00:41.933958 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 12:00:41.933958 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 12:00:41.933958 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 12:00:41.933958 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 12:00:41.933958 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 12:00:41.933958 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 12:00:41.933958 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:00:41.933958 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 12:00:41.933958 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:00:41.933958 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:41.933958 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:41.933958 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:41.933958 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:41.933958 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:41.935054 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:41.935054 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:41.935054 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:41.935054 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:41.935054 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:41.935054 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:41.935054 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 12:00:41.935054 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 12:00:41.935054 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 12:00:41.935054 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 12:00:41.935054 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:41.935054 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:41.935054 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:41.935054 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:41.935054 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:41.935054 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:41.935897 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:41.935897 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:41.935897 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:41.935897 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:41.935897 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:41.935897 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:41.935897 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:41.935897 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:41.935897 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:41.935897 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:41.935897 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 12:00:41.935897 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 12:00:41.935897 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 12:00:41.935897 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:00:41.938695 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 458/919] 10.222.0.49 () {56 vars in 1114 bytes} [Fri Sep 9 12:00:41 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 12:00:41.950465 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b2bab6-a5c3-4568-9da9-353a62b7dd3c None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:41.968694 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb07ff11-0bf3-40df-b38e-a75488dc6803 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:42.008715 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e295cd8-cb20-4e05-88c6-ed3d688d4842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:42.009622 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e295cd8-cb20-4e05-88c6-ed3d688d4842 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:42.010059 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e295cd8-cb20-4e05-88c6-ed3d688d4842 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:42.010780 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e295cd8-cb20-4e05-88c6-ed3d688d4842 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:42.024833 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e295cd8-cb20-4e05-88c6-ed3d688d4842 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:42.028064 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:42.028823 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e295cd8-cb20-4e05-88c6-ed3d688d4842 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:42.048675 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-4e295cd8-cb20-4e05-88c6-ed3d688d4842 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:00:42.048675 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:42.048675 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:42.048675 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:42.048675 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:42.048675 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:42.048675 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:42.048675 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:42.048675 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:42.048675 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:42.048675 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:42.048675 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:42.048675 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:00:42.048675 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:00:42.048675 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:00:42.048675 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:00:42.050279 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:42.050279 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:42.050279 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:42.050279 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:42.050279 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:42.050279 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:42.050279 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:42.050279 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:42.050279 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:42.050279 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:42.050279 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:42.050279 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:42.050279 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:42.050279 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:42.050279 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:42.050279 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:42.050279 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:00:42.051313 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:00:42.051313 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:00:42.051313 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:00:42.057455 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b2bab6-a5c3-4568-9da9-353a62b7dd3c None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:42.057594 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 462/920] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 12:00:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 12:00:42.064270 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e295cd8-cb20-4e05-88c6-ed3d688d4842 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:42.091989 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07c2527-526b-4325-8240-f216c9652fe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:42.092866 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07c2527-526b-4325-8240-f216c9652fe7 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:42.092979 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07c2527-526b-4325-8240-f216c9652fe7 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:42.093155 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07c2527-526b-4325-8240-f216c9652fe7 None admin] PATH_INFO: `/v3/services` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:42.093832 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07c2527-526b-4325-8240-f216c9652fe7 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:42.094136 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07c2527-526b-4325-8240-f216c9652fe7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:42.095534 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07c2527-526b-4325-8240-f216c9652fe7 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:42.105863 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 459/921] 10.222.0.49 () {56 vars in 1114 bytes} [Fri Sep 9 12:00:41 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:42.106724 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07c2527-526b-4325-8240-f216c9652fe7 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:42.162787 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:42.185704 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e295cd8-cb20-4e05-88c6-ed3d688d4842 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:42.211436 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07c2527-526b-4325-8240-f216c9652fe7 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:42.228741 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490f708a-9100-401b-9c35-88bdf55e05df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:42.229764 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490f708a-9100-401b-9c35-88bdf55e05df None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:42.230482 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490f708a-9100-401b-9c35-88bdf55e05df None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:42.230945 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490f708a-9100-401b-9c35-88bdf55e05df None admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:42.231726 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490f708a-9100-401b-9c35-88bdf55e05df None admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:42.232189 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490f708a-9100-401b-9c35-88bdf55e05df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:42.234114 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490f708a-9100-401b-9c35-88bdf55e05df None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:42.253132 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6720fcd-0041-45e9-8ed2-08a477b01bfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:42.253132 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6720fcd-0041-45e9-8ed2-08a477b01bfa None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:42.253132 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6720fcd-0041-45e9-8ed2-08a477b01bfa None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:42.253132 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6720fcd-0041-45e9-8ed2-08a477b01bfa None admin] PATH_INFO: `/v3/services` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:42.253685 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6720fcd-0041-45e9-8ed2-08a477b01bfa None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:42.253685 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6720fcd-0041-45e9-8ed2-08a477b01bfa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:42.253685 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6720fcd-0041-45e9-8ed2-08a477b01bfa None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:42.254378 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 463/922] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 9 12:00:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:42.255858 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490f708a-9100-401b-9c35-88bdf55e05df None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:42.263200 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 460/923] 10.222.0.49 () {56 vars in 1114 bytes} [Fri Sep 9 12:00:42 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:42.264148 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6720fcd-0041-45e9-8ed2-08a477b01bfa None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:42.279266 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:42.374872 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490f708a-9100-401b-9c35-88bdf55e05df None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:42.385607 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6720fcd-0041-45e9-8ed2-08a477b01bfa None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:42.412654 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91ff08c-755e-4453-a2f1-d2759527cbb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:42.413437 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91ff08c-755e-4453-a2f1-d2759527cbb1 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:42.413810 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91ff08c-755e-4453-a2f1-d2759527cbb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:42.414196 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91ff08c-755e-4453-a2f1-d2759527cbb1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:42.421749 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91ff08c-755e-4453-a2f1-d2759527cbb1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:42.423613 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91ff08c-755e-4453-a2f1-d2759527cbb1 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:42.433369 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19d9ea3-bc1a-417f-8fb2-f388f35e3056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:42.434202 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19d9ea3-bc1a-417f-8fb2-f388f35e3056 None admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:42.434633 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19d9ea3-bc1a-417f-8fb2-f388f35e3056 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:42.435071 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19d9ea3-bc1a-417f-8fb2-f388f35e3056 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:42.435997 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19d9ea3-bc1a-417f-8fb2-f388f35e3056 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:42.436808 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-a91ff08c-755e-4453-a2f1-d2759527cbb1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 12:00:42.436808 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:42.436808 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:42.436808 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:42.436808 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:42.436808 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:42.436808 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:42.436808 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:42.436808 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:42.436808 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:42.436808 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:42.436808 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:42.436808 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 12:00:42.436808 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 12:00:42.436808 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 12:00:42.436808 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 12:00:42.437800 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:42.437800 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:42.437800 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 12:00:42.437800 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:00:42.437800 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 12:00:42.437800 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:00:42.437800 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:42.437800 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:42.437800 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:42.437800 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:42.437800 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:42.437800 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:42.437800 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:42.437800 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:42.437800 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:42.437800 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:42.437800 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:42.437800 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:42.438745 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:42.438745 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:42.438745 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 12:00:42.438745 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 12:00:42.438745 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 12:00:42.438745 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 12:00:42.438745 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 12:00:42.438745 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 12:00:42.438745 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 12:00:42.438745 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:00:42.441868 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19d9ea3-bc1a-417f-8fb2-f388f35e3056 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:42.443408 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 464/924] 10.222.0.49 () {56 vars in 1117 bytes} [Fri Sep 9 12:00:42 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:42.445290 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91ff08c-755e-4453-a2f1-d2759527cbb1 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:42.448325 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:42.471825 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 461/925] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:00:42 2022] POST /identity/v3/endpoints => generated 346 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:00:42.574680 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91ff08c-755e-4453-a2f1-d2759527cbb1 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:42.621759 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e21d0f9-405d-4d1d-bde7-2c3bff43d8fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:42.622700 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e21d0f9-405d-4d1d-bde7-2c3bff43d8fb None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:42.623201 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e21d0f9-405d-4d1d-bde7-2c3bff43d8fb None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:42.623649 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e21d0f9-405d-4d1d-bde7-2c3bff43d8fb None admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:42.624591 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e21d0f9-405d-4d1d-bde7-2c3bff43d8fb None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:42.625071 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e21d0f9-405d-4d1d-bde7-2c3bff43d8fb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:42.626696 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e21d0f9-405d-4d1d-bde7-2c3bff43d8fb None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:42.649286 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 465/926] 10.222.0.49 () {56 vars in 1149 bytes} [Fri Sep 9 12:00:42 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:00:42.654810 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19d9ea3-bc1a-417f-8fb2-f388f35e3056 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:42.727137 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:42.773379 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19d9ea3-bc1a-417f-8fb2-f388f35e3056 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:42.788951 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1518834f-dfcc-4666-b6c1-99f46bc054a1 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:42.789555 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1518834f-dfcc-4666-b6c1-99f46bc054a1 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:42.790029 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1518834f-dfcc-4666-b6c1-99f46bc054a1 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:42.792924 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 466/927] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:00:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 12:00:42.797240 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2968a0-38da-4a4a-9fc6-8ad69f486e18 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:42.797752 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2968a0-38da-4a4a-9fc6-8ad69f486e18 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:42.798237 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2968a0-38da-4a4a-9fc6-8ad69f486e18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:42.811324 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb2d981-1bb4-49ae-a372-f6359d870b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:42.811957 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb2d981-1bb4-49ae-a372-f6359d870b7d None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:42.812231 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb2d981-1bb4-49ae-a372-f6359d870b7d None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:42.812550 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb2d981-1bb4-49ae-a372-f6359d870b7d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:42.818416 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-0e2968a0-38da-4a4a-9fc6-8ad69f486e18 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:00:42.819907 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-0e2968a0-38da-4a4a-9fc6-8ad69f486e18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:42.826437 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb2d981-1bb4-49ae-a372-f6359d870b7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:42.827925 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb2d981-1bb4-49ae-a372-f6359d870b7d None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:42.853148 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-9fb2d981-1bb4-49ae-a372-f6359d870b7d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:00:42.853148 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:42.853148 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:42.853148 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:42.853148 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:42.853148 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:42.853148 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:42.853148 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:42.853148 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:42.853148 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:42.853148 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:42.853148 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:42.853148 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:00:42.853148 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:00:42.853148 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:00:42.853148 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:00:42.854160 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:42.854160 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:42.854160 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:42.854160 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:42.854160 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:42.854160 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:42.854160 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:42.854160 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:42.854160 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:42.854160 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:42.854160 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:42.854160 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:42.854160 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:42.854160 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:42.854160 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:42.854160 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:42.854160 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:00:42.855656 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:00:42.855656 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:00:42.855656 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:00:42.855656 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 462/928] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 12:00:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:42.859996 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb2d981-1bb4-49ae-a372-f6359d870b7d None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:42.885118 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:42.911226 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 467/929] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:00:42 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 12:00:42.990213 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb2d981-1bb4-49ae-a372-f6359d870b7d None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:43.004430 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:43.027325 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2201a4e9-1185-4b57-b28f-1c2567d527f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:43.027933 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2201a4e9-1185-4b57-b28f-1c2567d527f9 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:43.028207 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2201a4e9-1185-4b57-b28f-1c2567d527f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:43.028528 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2201a4e9-1185-4b57-b28f-1c2567d527f9 None admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:43.029162 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2201a4e9-1185-4b57-b28f-1c2567d527f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:43.029460 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2201a4e9-1185-4b57-b28f-1c2567d527f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:43.030933 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2201a4e9-1185-4b57-b28f-1c2567d527f9 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:43.035786 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bfbf93-5119-4fc9-b63d-eb09bfce7a6a None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:43.036072 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bfbf93-5119-4fc9-b63d-eb09bfce7a6a None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:43.036350 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bfbf93-5119-4fc9-b63d-eb09bfce7a6a None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:43.037957 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 468/930] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:00:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 12:00:43.043116 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79ddcb6-6de2-4437-9850-209f85ebe1f6 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:43.043401 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79ddcb6-6de2-4437-9850-209f85ebe1f6 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:43.043674 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79ddcb6-6de2-4437-9850-209f85ebe1f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:43.049603 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 463/931] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 9 12:00:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:43.056563 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2201a4e9-1185-4b57-b28f-1c2567d527f9 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:43.066098 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-f79ddcb6-6de2-4437-9850-209f85ebe1f6 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:00:43.067344 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-f79ddcb6-6de2-4437-9850-209f85ebe1f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:43.143785 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 469/932] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:00:43 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 12:00:43.176731 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2201a4e9-1185-4b57-b28f-1c2567d527f9 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:43.180442 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79ddcb6-6de2-4437-9850-209f85ebe1f6 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:43.220335 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8968409d-f855-41e1-8e67-7668ceb5ff7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:43.221132 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8968409d-f855-41e1-8e67-7668ceb5ff7a None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:43.221380 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8968409d-f855-41e1-8e67-7668ceb5ff7a None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:43.221813 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8968409d-f855-41e1-8e67-7668ceb5ff7a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:43.230588 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8968409d-f855-41e1-8e67-7668ceb5ff7a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:43.232999 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8968409d-f855-41e1-8e67-7668ceb5ff7a None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:43.249734 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-8968409d-f855-41e1-8e67-7668ceb5ff7a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:00:43.249734 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:43.249734 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:43.249734 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:43.249734 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:43.249734 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:43.249734 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:43.249734 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:43.249734 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:43.249734 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:43.249734 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:43.249734 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:43.249734 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 12:00:43.249734 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 12:00:43.249734 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 12:00:43.250831 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 12:00:43.250831 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:43.250831 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:43.250831 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:43.250831 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:43.250831 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:43.250831 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:43.250831 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:43.250831 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:43.250831 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:43.250831 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:43.250831 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:43.250831 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:43.250831 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:43.250831 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:43.250831 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:43.250831 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:43.251957 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 12:00:43.251957 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 12:00:43.251957 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 12:00:43.251957 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 12:00:43.251957 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 12:00:43.251957 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 12:00:43.251957 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:00:43.251957 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:00:43.259712 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 464/933] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 9 12:00:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 12:00:43.261423 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8968409d-f855-41e1-8e67-7668ceb5ff7a None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:43.295648 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79ddcb6-6de2-4437-9850-209f85ebe1f6 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:43.302883 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:43.314872 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:43.336621 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c80fc3f-f7b9-4804-b22a-5ac69fdb9f20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:43.337261 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c80fc3f-f7b9-4804-b22a-5ac69fdb9f20 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:43.337542 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c80fc3f-f7b9-4804-b22a-5ac69fdb9f20 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:43.337816 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c80fc3f-f7b9-4804-b22a-5ac69fdb9f20 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:43.338400 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c80fc3f-f7b9-4804-b22a-5ac69fdb9f20 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:43.339817 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c80fc3f-f7b9-4804-b22a-5ac69fdb9f20 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:43.350951 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-6c80fc3f-f7b9-4804-b22a-5ac69fdb9f20 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 12:00:43.350951 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:43.350951 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 12:00:43.350951 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 12:00:43.350951 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 12:00:43.350951 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 12:00:43.350951 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 12:00:43.350951 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 12:00:43.350951 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 12:00:43.350951 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:00:43.350951 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 12:00:43.350951 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:00:43.350951 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:43.350951 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:43.350951 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:43.350951 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:43.350951 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:43.351976 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:43.351976 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:43.351976 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:43.351976 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:43.351976 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:43.351976 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:43.351976 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 12:00:43.351976 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 12:00:43.351976 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 12:00:43.351976 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 12:00:43.351976 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:43.351976 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:43.351976 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:43.351976 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:43.351976 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:43.351976 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:43.352805 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:43.352805 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:43.352805 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:43.352805 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:43.352805 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:43.352805 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:43.352805 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:43.352805 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:43.352805 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:43.352805 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:43.352805 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 12:00:43.352805 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 12:00:43.352805 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 12:00:43.352805 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:00:43.352805 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 470/934] 10.222.0.49 () {56 vars in 1126 bytes} [Fri Sep 9 12:00:43 2022] GET /identity/v3/services/object-store => generated 93 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 12:00:43.356962 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c80fc3f-f7b9-4804-b22a-5ac69fdb9f20 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:43.397234 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8968409d-f855-41e1-8e67-7668ceb5ff7a None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:43.435459 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d88612-61b2-4b94-a718-07c5e99e35b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:43.436376 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d88612-61b2-4b94-a718-07c5e99e35b2 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:43.436840 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d88612-61b2-4b94-a718-07c5e99e35b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:43.437122 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d88612-61b2-4b94-a718-07c5e99e35b2 None admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:43.437813 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d88612-61b2-4b94-a718-07c5e99e35b2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:43.438121 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d88612-61b2-4b94-a718-07c5e99e35b2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:43.439974 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d88612-61b2-4b94-a718-07c5e99e35b2 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:43.453717 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 465/935] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Sep 9 12:00:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:43.454801 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d88612-61b2-4b94-a718-07c5e99e35b2 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:43.482684 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c80fc3f-f7b9-4804-b22a-5ac69fdb9f20 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:43.521330 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723f1f44-2665-4334-87af-eb9cb07e282a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:43.521861 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723f1f44-2665-4334-87af-eb9cb07e282a None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:43.522139 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723f1f44-2665-4334-87af-eb9cb07e282a None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:43.522412 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723f1f44-2665-4334-87af-eb9cb07e282a None admin] PATH_INFO: `/v3/services` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:43.523072 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723f1f44-2665-4334-87af-eb9cb07e282a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:43.523373 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723f1f44-2665-4334-87af-eb9cb07e282a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:43.525088 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723f1f44-2665-4334-87af-eb9cb07e282a None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:43.534394 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 471/936] 10.222.0.49 () {56 vars in 1122 bytes} [Fri Sep 9 12:00:43 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:00:43.534394 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723f1f44-2665-4334-87af-eb9cb07e282a None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:43.553597 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:43.564313 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d88612-61b2-4b94-a718-07c5e99e35b2 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:43.584660 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:43.600866 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c010667-83e4-4d7f-8a81-a85811b16fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:43.601497 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c010667-83e4-4d7f-8a81-a85811b16fa2 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:43.601885 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c010667-83e4-4d7f-8a81-a85811b16fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:43.602162 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c010667-83e4-4d7f-8a81-a85811b16fa2 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:43.602744 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c010667-83e4-4d7f-8a81-a85811b16fa2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:43.604140 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c010667-83e4-4d7f-8a81-a85811b16fa2 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:43.613083 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-9c010667-83e4-4d7f-8a81-a85811b16fa2 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 12:00:43.613083 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:43.613083 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 12:00:43.613083 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 12:00:43.613083 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 12:00:43.613083 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 12:00:43.613083 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 12:00:43.613083 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 12:00:43.613083 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 12:00:43.613083 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:00:43.613083 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 12:00:43.613083 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:00:43.613083 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:43.613083 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:43.613083 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:43.613083 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:43.614175 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:43.614175 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:43.614175 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:43.614175 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:43.614175 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:43.614175 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:43.614175 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:43.614175 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 12:00:43.614175 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 12:00:43.614175 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 12:00:43.614175 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 12:00:43.614175 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:43.614175 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:43.614175 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:43.614175 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:43.614175 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:43.614175 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:43.615296 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:43.615296 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:43.615296 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:43.615296 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:43.615296 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:43.615296 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:43.615296 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:43.615296 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:43.615296 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:43.615296 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:43.615296 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 12:00:43.615296 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 12:00:43.615296 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 12:00:43.615296 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:00:43.615296 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 466/937] 10.222.0.49 () {56 vars in 1132 bytes} [Fri Sep 9 12:00:43 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:43.616094 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c010667-83e4-4d7f-8a81-a85811b16fa2 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:43.651697 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723f1f44-2665-4334-87af-eb9cb07e282a None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:43.687271 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232e3671-cf2b-4292-98e7-d00d4cff25b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:43.687935 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232e3671-cf2b-4292-98e7-d00d4cff25b2 None admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:43.688271 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232e3671-cf2b-4292-98e7-d00d4cff25b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:43.688547 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232e3671-cf2b-4292-98e7-d00d4cff25b2 None admin] PATH_INFO: `/v3/projects/fe863fcd75f1406ebbffc5016078f73e/users/e368b4e668c54e25bff80a8e8a8c3588/roles/db537712f45d4e6980592d3ddefb312a` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:43.691815 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232e3671-cf2b-4292-98e7-d00d4cff25b2 None admin] RBAC: Authorizing `identity:create_grant(project_id=fe863fcd75f1406ebbffc5016078f73e, user_id=e368b4e668c54e25bff80a8e8a8c3588, role_id=db537712f45d4e6980592d3ddefb312a)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:43.693254 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232e3671-cf2b-4292-98e7-d00d4cff25b2 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:43.715190 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 472/938] 10.222.0.49 () {58 vars in 1439 bytes} [Fri Sep 9 12:00:43 2022] PUT /identity/v3/projects/fe863fcd75f1406ebbffc5016078f73e/users/e368b4e668c54e25bff80a8e8a8c3588/roles/db537712f45d4e6980592d3ddefb312a => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:00:43.716235 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232e3671-cf2b-4292-98e7-d00d4cff25b2 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:43.732404 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c010667-83e4-4d7f-8a81-a85811b16fa2 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:43.785738 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915527d4-206f-485c-90cc-6c06d4602c24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:43.786438 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915527d4-206f-485c-90cc-6c06d4602c24 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:43.786834 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915527d4-206f-485c-90cc-6c06d4602c24 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:43.787543 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915527d4-206f-485c-90cc-6c06d4602c24 None admin] PATH_INFO: `/v3/services` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:43.788067 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915527d4-206f-485c-90cc-6c06d4602c24 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:43.788464 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915527d4-206f-485c-90cc-6c06d4602c24 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:43.789958 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915527d4-206f-485c-90cc-6c06d4602c24 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:43.796493 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 467/939] 10.222.0.49 () {56 vars in 1122 bytes} [Fri Sep 9 12:00:43 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:43.802790 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915527d4-206f-485c-90cc-6c06d4602c24 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:43.833083 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:43.851959 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232e3671-cf2b-4292-98e7-d00d4cff25b2 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:43.855581 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:43.895229 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03b7852-c472-4104-82a7-e14f29bb4a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:43.895852 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03b7852-c472-4104-82a7-e14f29bb4a22 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:43.896128 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03b7852-c472-4104-82a7-e14f29bb4a22 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:43.896405 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03b7852-c472-4104-82a7-e14f29bb4a22 None admin] PATH_INFO: `/v3/services` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:43.897041 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03b7852-c472-4104-82a7-e14f29bb4a22 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:43.897344 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03b7852-c472-4104-82a7-e14f29bb4a22 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:43.898731 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03b7852-c472-4104-82a7-e14f29bb4a22 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:43.904296 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 473/940] 10.222.0.49 () {56 vars in 1126 bytes} [Fri Sep 9 12:00:43 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:43.912150 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03b7852-c472-4104-82a7-e14f29bb4a22 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:43.949199 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915527d4-206f-485c-90cc-6c06d4602c24 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:43.994885 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c093bc-e630-49cf-a22a-355c72cd27bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:43.996247 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c093bc-e630-49cf-a22a-355c72cd27bf None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:43.996957 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c093bc-e630-49cf-a22a-355c72cd27bf None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:43.997404 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c093bc-e630-49cf-a22a-355c72cd27bf None admin] PATH_INFO: `/v3/endpoints` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:43.998481 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c093bc-e630-49cf-a22a-355c72cd27bf None admin] RBAC: Adding query filter params (interface=public, service_id=526d33a9603a4b869299c9b155a834e1, region_id=RegionOne) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:43.998958 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c093bc-e630-49cf-a22a-355c72cd27bf None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:44.001259 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c093bc-e630-49cf-a22a-355c72cd27bf None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:44.008164 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 468/941] 10.222.0.49 () {56 vars in 1251 bytes} [Fri Sep 9 12:00:43 2022] GET /identity/v3/endpoints?service_id=526d33a9603a4b869299c9b155a834e1&interface=public®ion_id=RegionOne => generated 194 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:44.013851 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c093bc-e630-49cf-a22a-355c72cd27bf None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:44.038614 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03b7852-c472-4104-82a7-e14f29bb4a22 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:44.084042 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb99fcf0-5d27-4952-afc7-f0e5c9e9d891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:44.084741 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb99fcf0-5d27-4952-afc7-f0e5c9e9d891 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:44.085088 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb99fcf0-5d27-4952-afc7-f0e5c9e9d891 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:44.085419 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb99fcf0-5d27-4952-afc7-f0e5c9e9d891 None admin] PATH_INFO: `/v3/services` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:44.086113 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb99fcf0-5d27-4952-afc7-f0e5c9e9d891 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:44.086474 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb99fcf0-5d27-4952-afc7-f0e5c9e9d891 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:44.088113 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb99fcf0-5d27-4952-afc7-f0e5c9e9d891 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:44.098990 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 474/942] 10.222.0.49 () {56 vars in 1126 bytes} [Fri Sep 9 12:00:43 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:44.102358 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb99fcf0-5d27-4952-afc7-f0e5c9e9d891 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:44.123300 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:44.129791 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c093bc-e630-49cf-a22a-355c72cd27bf None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:44.142633 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:44.168957 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1dadd8-ecdf-4f2e-88cc-3f0b6d6acfe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:44.169566 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1dadd8-ecdf-4f2e-88cc-3f0b6d6acfe5 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:44.169845 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1dadd8-ecdf-4f2e-88cc-3f0b6d6acfe5 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:44.170113 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1dadd8-ecdf-4f2e-88cc-3f0b6d6acfe5 None admin] PATH_INFO: `/v3/services` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:44.170713 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1dadd8-ecdf-4f2e-88cc-3f0b6d6acfe5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:44.172448 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1dadd8-ecdf-4f2e-88cc-3f0b6d6acfe5 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:44.186262 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 469/943] 10.222.0.49 () {56 vars in 1087 bytes} [Fri Sep 9 12:00:44 2022] GET /identity/v3/services => generated 1497 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 12:00:44.228433 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb99fcf0-5d27-4952-afc7-f0e5c9e9d891 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:44.266889 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360cd167-cef6-4ea6-b602-53cac7b3f690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:44.268519 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360cd167-cef6-4ea6-b602-53cac7b3f690 None admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:44.269219 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360cd167-cef6-4ea6-b602-53cac7b3f690 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:44.269832 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360cd167-cef6-4ea6-b602-53cac7b3f690 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:44.270780 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360cd167-cef6-4ea6-b602-53cac7b3f690 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:44.272463 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360cd167-cef6-4ea6-b602-53cac7b3f690 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:44.302368 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 475/944] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:00:44 2022] POST /identity/v3/endpoints => generated 347 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:00:44.828302 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d684248-2269-476b-86ed-40eb39477f51 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:44.828805 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d684248-2269-476b-86ed-40eb39477f51 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:44.828911 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d684248-2269-476b-86ed-40eb39477f51 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:44.830989 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 470/945] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:00:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:00:44.836363 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e279d7eb-2591-4ab2-9614-c82d901770f2 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:44.836647 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e279d7eb-2591-4ab2-9614-c82d901770f2 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:44.836917 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e279d7eb-2591-4ab2-9614-c82d901770f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:44.855166 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-e279d7eb-2591-4ab2-9614-c82d901770f2 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:00:44.856345 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-e279d7eb-2591-4ab2-9614-c82d901770f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:44.930200 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 476/946] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:00:44 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 12:00:45.202148 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d684248-2269-476b-86ed-40eb39477f51 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:45.309151 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d684248-2269-476b-86ed-40eb39477f51 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:45.343620 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835f4722-b4a6-4587-a767-338147870a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:45.344523 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835f4722-b4a6-4587-a767-338147870a01 None admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:45.345023 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835f4722-b4a6-4587-a767-338147870a01 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:45.345512 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835f4722-b4a6-4587-a767-338147870a01 None admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:45.346923 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835f4722-b4a6-4587-a767-338147870a01 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:45.348571 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835f4722-b4a6-4587-a767-338147870a01 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:45.351377 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: WARNING keystone.server.flask.common [None req-835f4722-b4a6-4587-a767-338147870a01 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 09 12:00:45.371949 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.sql.core [None req-835f4722-b4a6-4587-a767-338147870a01 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 09 12:00:45.371949 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 09 12:00:45.371949 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [parameters: {'id': 'faa7c884f6da4b268bfd5c5c53790d84', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 09 12:00:45.371949 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81590) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 09 12:00:45.389898 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-835f4722-b4a6-4587-a767-338147870a01 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id faa7c884f6da4b268bfd5c5c53790d84.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id faa7c884f6da4b268bfd5c5c53790d84. Sep 09 12:00:45.389898 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:45.389898 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 09 12:00:45.389898 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 09 12:00:45.389898 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 09 12:00:45.389898 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 09 12:00:45.389898 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 09 12:00:45.389898 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application result = self._query(query) Sep 09 12:00:45.389898 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 09 12:00:45.389898 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application conn.query(q) Sep 09 12:00:45.389898 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 09 12:00:45.389898 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 09 12:00:45.389898 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 09 12:00:45.390759 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application result.read() Sep 09 12:00:45.390759 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 09 12:00:45.390759 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 09 12:00:45.390759 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 09 12:00:45.390759 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 09 12:00:45.390759 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 09 12:00:45.390759 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 09 12:00:45.390759 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 09 12:00:45.390759 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 09 12:00:45.390759 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 09 12:00:45.390759 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:00:45.390759 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 09 12:00:45.390759 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:00:45.390759 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:45.390759 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 09 12:00:45.390759 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 09 12:00:45.390759 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 09 12:00:45.390759 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 09 12:00:45.391769 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 09 12:00:45.391769 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application next(self.gen) Sep 09 12:00:45.391769 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 09 12:00:45.391769 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application yield resource Sep 09 12:00:45.391769 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 09 12:00:45.391769 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application next(self.gen) Sep 09 12:00:45.391769 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 09 12:00:45.391769 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application self.session.rollback() Sep 09 12:00:45.391769 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 09 12:00:45.391769 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application self.force_reraise() Sep 09 12:00:45.391769 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 09 12:00:45.391769 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise self.value Sep 09 12:00:45.391769 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 09 12:00:45.391769 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 09 12:00:45.391769 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 09 12:00:45.391769 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application session.commit() Sep 09 12:00:45.391769 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 09 12:00:45.392670 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 09 12:00:45.392670 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 09 12:00:45.392670 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application self._prepare_impl() Sep 09 12:00:45.392670 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 09 12:00:45.392670 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application self.session.flush() Sep 09 12:00:45.392670 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 09 12:00:45.392670 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application self._flush(objects) Sep 09 12:00:45.392670 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 09 12:00:45.392670 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 09 12:00:45.392670 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 09 12:00:45.392670 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application compat.raise_( Sep 09 12:00:45.392670 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 09 12:00:45.392670 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception Sep 09 12:00:45.392670 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 09 12:00:45.392670 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application flush_context.execute() Sep 09 12:00:45.392670 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 09 12:00:45.392670 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rec.execute(self) Sep 09 12:00:45.394499 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 09 12:00:45.394499 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 09 12:00:45.394499 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 09 12:00:45.394499 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 09 12:00:45.394499 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 09 12:00:45.394499 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 09 12:00:45.394499 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 09 12:00:45.394499 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 09 12:00:45.394499 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 09 12:00:45.394499 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 09 12:00:45.394499 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 09 12:00:45.394499 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 09 12:00:45.394499 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 09 12:00:45.394499 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 09 12:00:45.394499 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 09 12:00:45.396571 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 09 12:00:45.396571 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 09 12:00:45.396571 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception Sep 09 12:00:45.396571 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 09 12:00:45.396571 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 09 12:00:45.396571 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 09 12:00:45.396571 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 09 12:00:45.396571 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 09 12:00:45.396571 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application result = self._query(query) Sep 09 12:00:45.396571 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 09 12:00:45.396571 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application conn.query(q) Sep 09 12:00:45.396571 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 09 12:00:45.396571 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 09 12:00:45.396571 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 09 12:00:45.396571 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application result.read() Sep 09 12:00:45.396571 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 09 12:00:45.396571 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 09 12:00:45.398711 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 09 12:00:45.398711 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 09 12:00:45.398711 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 09 12:00:45.398711 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 09 12:00:45.398711 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 09 12:00:45.398711 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 09 12:00:45.398711 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 09 12:00:45.398711 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 09 12:00:45.398711 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [parameters: {'id': 'faa7c884f6da4b268bfd5c5c53790d84', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 09 12:00:45.398711 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 09 12:00:45.398711 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:00:45.398711 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 12:00:45.398711 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:00:45.398711 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:45.398711 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 09 12:00:45.400613 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 09 12:00:45.400613 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 09 12:00:45.400613 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 09 12:00:45.400613 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 09 12:00:45.400613 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:00:45.400613 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 12:00:45.400613 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:00:45.400613 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:45.400613 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:45.400613 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:45.400613 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:45.400613 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:45.400613 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:45.400613 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:45.400613 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:45.400613 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:45.400613 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:45.403414 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:45.403414 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 09 12:00:45.403414 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 09 12:00:45.403414 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:45.403414 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:45.403414 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 09 12:00:45.403414 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 09 12:00:45.403414 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id faa7c884f6da4b268bfd5c5c53790d84. Sep 09 12:00:45.403414 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:00:45.405507 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 471/947] 10.222.0.49 () {60 vars in 1140 bytes} [Fri Sep 9 12:00:45 2022] POST /identity/v3/projects => generated 262 bytes in 203 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:45.411083 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e279d7eb-2591-4ab2-9614-c82d901770f2 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:45.463415 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:45.516970 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e279d7eb-2591-4ab2-9614-c82d901770f2 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:45.553753 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b5d963-bc14-42be-976a-1922b6316462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:45.554746 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b5d963-bc14-42be-976a-1922b6316462 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:45.555258 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b5d963-bc14-42be-976a-1922b6316462 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:45.555792 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b5d963-bc14-42be-976a-1922b6316462 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:45.563194 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b5d963-bc14-42be-976a-1922b6316462 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:45.565219 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b5d963-bc14-42be-976a-1922b6316462 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:45.577116 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-30b5d963-bc14-42be-976a-1922b6316462 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:00:45.577116 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:45.577116 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:45.577116 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:45.577116 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:45.577116 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:45.577116 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:45.577116 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:45.577116 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:45.577116 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:45.577116 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:45.577116 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:45.577116 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 12:00:45.577116 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 12:00:45.577116 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 12:00:45.578009 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 12:00:45.578009 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:45.578009 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:45.578009 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:45.578009 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:45.578009 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:45.578009 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:45.578009 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:45.578009 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:45.578009 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:45.578009 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:45.578009 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:45.578009 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:45.578009 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:45.578009 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:45.578009 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:45.578009 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:45.578891 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 12:00:45.578891 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 12:00:45.578891 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 12:00:45.578891 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 12:00:45.578891 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 12:00:45.578891 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 12:00:45.578891 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:00:45.578891 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:00:45.581923 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 477/948] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 9 12:00:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:45.587682 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835f4722-b4a6-4587-a767-338147870a01 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:45.603423 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:45.708170 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835f4722-b4a6-4587-a767-338147870a01 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:45.747306 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84475ad0-e231-4822-bc78-ff37970c0305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:45.748278 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84475ad0-e231-4822-bc78-ff37970c0305 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:45.748758 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84475ad0-e231-4822-bc78-ff37970c0305 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:45.749223 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84475ad0-e231-4822-bc78-ff37970c0305 None admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:45.750110 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84475ad0-e231-4822-bc78-ff37970c0305 None admin] RBAC: Adding query filter params (name=service) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:45.750625 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84475ad0-e231-4822-bc78-ff37970c0305 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:45.752725 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84475ad0-e231-4822-bc78-ff37970c0305 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:45.773898 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 472/949] 10.222.0.49 () {56 vars in 1112 bytes} [Fri Sep 9 12:00:45 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:46.025459 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb0b337-6da1-49fa-ae9f-dc353fc0bb67 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:46.025835 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb0b337-6da1-49fa-ae9f-dc353fc0bb67 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:46.026038 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb0b337-6da1-49fa-ae9f-dc353fc0bb67 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:46.027603 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 478/950] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:00:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:00:46.033975 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107edb23-13b4-4c0b-8ab3-3b8d2b4e124c None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:46.034853 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107edb23-13b4-4c0b-8ab3-3b8d2b4e124c None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:46.035385 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107edb23-13b4-4c0b-8ab3-3b8d2b4e124c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:46.055072 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-107edb23-13b4-4c0b-8ab3-3b8d2b4e124c None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:00:46.056917 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-107edb23-13b4-4c0b-8ab3-3b8d2b4e124c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:46.121850 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:46.137819 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 473/951] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:00:46 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 12:00:46.409759 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb0b337-6da1-49fa-ae9f-dc353fc0bb67 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:46.467247 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96668141-5d64-493b-bf85-d34485bfbb37 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:46.467621 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96668141-5d64-493b-bf85-d34485bfbb37 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:46.467804 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96668141-5d64-493b-bf85-d34485bfbb37 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:46.469547 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 474/952] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:00:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:00:46.476278 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4fe109-8105-42cc-9c39-61209956932c None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:46.476629 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4fe109-8105-42cc-9c39-61209956932c None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:46.476823 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4fe109-8105-42cc-9c39-61209956932c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:46.496825 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:46.497785 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-dd4fe109-8105-42cc-9c39-61209956932c None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:00:46.499906 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-dd4fe109-8105-42cc-9c39-61209956932c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:46.528903 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb0b337-6da1-49fa-ae9f-dc353fc0bb67 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:46.573516 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903db148-0db4-4771-b971-fdf4b7fc959a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:46.574160 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903db148-0db4-4771-b971-fdf4b7fc959a None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:46.574437 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903db148-0db4-4771-b971-fdf4b7fc959a None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:46.574704 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903db148-0db4-4771-b971-fdf4b7fc959a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:46.584332 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 475/953] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:00:46 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 12:00:46.591104 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903db148-0db4-4771-b971-fdf4b7fc959a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:00:46.592599 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-903db148-0db4-4771-b971-fdf4b7fc959a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 12:00:46.592599 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:46.592599 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:46.592599 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:46.592599 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:46.592599 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:46.592599 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:46.592599 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:46.592599 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:46.592599 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:46.592599 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:46.592599 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:46.592599 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:00:46.592599 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:00:46.592599 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:00:46.592599 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:00:46.593654 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:00:46.593654 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:00:46.593654 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:00:46.593654 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:00:46.593654 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:46.593654 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:46.593654 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:46.593654 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:46.593654 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:46.593654 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:46.593654 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:46.593654 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:46.593654 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:46.593654 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:46.593654 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:46.593654 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:46.594561 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:46.594561 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:46.594561 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:46.594561 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:46.594561 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:00:46.594561 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:00:46.594561 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:00:46.594561 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:00:46.594561 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:00:46.594561 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:00:46.594561 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 12:00:46.594561 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:00:46.595265 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8b9428-7414-4cac-af82-d8dec38b3747 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:46.595265 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8b9428-7414-4cac-af82-d8dec38b3747 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:46.595265 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8b9428-7414-4cac-af82-d8dec38b3747 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:46.595461 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 476/954] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:00:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:00:46.595536 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 479/955] 10.222.0.49 () {56 vars in 1120 bytes} [Fri Sep 9 12:00:46 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:46.600159 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8b9428-7414-4cac-af82-d8dec38b3747 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:46.601854 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f3bbf9-c43f-4e1d-9e17-0d47911411c3 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:46.602287 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f3bbf9-c43f-4e1d-9e17-0d47911411c3 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:46.602886 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f3bbf9-c43f-4e1d-9e17-0d47911411c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:46.622142 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-68f3bbf9-c43f-4e1d-9e17-0d47911411c3 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:00:46.623682 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-68f3bbf9-c43f-4e1d-9e17-0d47911411c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:46.704042 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 480/956] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:00:46 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 12:00:46.709755 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8b9428-7414-4cac-af82-d8dec38b3747 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:46.734137 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:46.745526 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766aeb43-64f5-4af6-81ca-1097b8c7b3c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:46.746126 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766aeb43-64f5-4af6-81ca-1097b8c7b3c0 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:46.746403 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766aeb43-64f5-4af6-81ca-1097b8c7b3c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:46.746670 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766aeb43-64f5-4af6-81ca-1097b8c7b3c0 None admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:46.747357 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766aeb43-64f5-4af6-81ca-1097b8c7b3c0 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:46.747654 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766aeb43-64f5-4af6-81ca-1097b8c7b3c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:46.749085 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766aeb43-64f5-4af6-81ca-1097b8c7b3c0 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:46.763197 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 477/957] 10.222.0.49 () {56 vars in 1115 bytes} [Fri Sep 9 12:00:46 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:00:46.765073 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f3bbf9-c43f-4e1d-9e17-0d47911411c3 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:46.829750 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:46.868750 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f3bbf9-c43f-4e1d-9e17-0d47911411c3 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:46.869077 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766aeb43-64f5-4af6-81ca-1097b8c7b3c0 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:46.906004 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6441df6e-ac1d-4365-b3b4-21e85649ddfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:46.906642 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6441df6e-ac1d-4365-b3b4-21e85649ddfe None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:46.906938 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6441df6e-ac1d-4365-b3b4-21e85649ddfe None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:46.907207 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6441df6e-ac1d-4365-b3b4-21e85649ddfe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:46.918408 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6441df6e-ac1d-4365-b3b4-21e85649ddfe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:46.919995 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6441df6e-ac1d-4365-b3b4-21e85649ddfe None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:46.935363 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-6441df6e-ac1d-4365-b3b4-21e85649ddfe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:00:46.935363 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:46.935363 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:46.935363 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:46.935363 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:46.935363 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:46.935363 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:46.935363 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:46.935363 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:46.935363 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:46.935363 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:46.935363 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:46.935363 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:00:46.935363 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:00:46.935363 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:00:46.935363 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:00:46.936404 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:46.936404 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:46.936404 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:46.936404 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:46.936404 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:46.936404 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:46.936404 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:46.936404 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:46.936404 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:46.936404 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:46.936404 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:46.936404 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:46.936404 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:46.936404 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:46.936404 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:46.936404 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:46.936404 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:00:46.937305 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:00:46.937305 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:00:46.937305 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:00:46.937305 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 481/958] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 12:00:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:46.947961 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6441df6e-ac1d-4365-b3b4-21e85649ddfe None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:46.991085 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766aeb43-64f5-4af6-81ca-1097b8c7b3c0 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:47.029434 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad5103a-b115-4cef-8755-cd5a663ce6fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:47.030068 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad5103a-b115-4cef-8755-cd5a663ce6fa None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:47.030345 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad5103a-b115-4cef-8755-cd5a663ce6fa None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:47.030612 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad5103a-b115-4cef-8755-cd5a663ce6fa None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:47.031278 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad5103a-b115-4cef-8755-cd5a663ce6fa None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:47.032986 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad5103a-b115-4cef-8755-cd5a663ce6fa None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:47.042665 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-5ad5103a-b115-4cef-8755-cd5a663ce6fa None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 12:00:47.042665 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:47.042665 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 12:00:47.042665 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 12:00:47.042665 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 12:00:47.042665 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 12:00:47.042665 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 12:00:47.042665 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 12:00:47.042665 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 12:00:47.042665 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:00:47.042665 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 12:00:47.042665 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:00:47.042665 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:47.042665 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:47.042665 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:47.042665 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:47.042665 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:47.044365 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:47.044365 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:47.044365 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:47.044365 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:47.044365 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:47.044365 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:47.044365 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 12:00:47.044365 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 12:00:47.044365 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 12:00:47.044365 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 12:00:47.044365 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:47.044365 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:47.044365 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:47.044365 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:47.044365 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:47.044365 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:47.045206 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:47.045206 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:47.045206 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:47.045206 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:47.045206 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:47.045206 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:47.045206 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:47.045206 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:47.045206 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:47.045206 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:47.045206 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 12:00:47.045206 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 12:00:47.045206 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 12:00:47.045206 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:00:47.045206 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 478/959] 10.222.0.49 () {56 vars in 1126 bytes} [Fri Sep 9 12:00:46 2022] GET /identity/v3/services/object-store => generated 93 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 12:00:47.046045 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad5103a-b115-4cef-8755-cd5a663ce6fa None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:47.069243 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6441df6e-ac1d-4365-b3b4-21e85649ddfe None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:47.088083 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:47.104481 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:47.107780 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1001b7-6dc1-47c6-ad79-0c4790cf9c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:47.108492 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1001b7-6dc1-47c6-ad79-0c4790cf9c7b None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:47.108776 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1001b7-6dc1-47c6-ad79-0c4790cf9c7b None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:47.109095 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1001b7-6dc1-47c6-ad79-0c4790cf9c7b None admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:47.109762 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1001b7-6dc1-47c6-ad79-0c4790cf9c7b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:47.110150 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1001b7-6dc1-47c6-ad79-0c4790cf9c7b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:47.112083 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1001b7-6dc1-47c6-ad79-0c4790cf9c7b None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:47.124929 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 482/960] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 9 12:00:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:47.130875 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1001b7-6dc1-47c6-ad79-0c4790cf9c7b None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:47.165270 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad5103a-b115-4cef-8755-cd5a663ce6fa None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:47.200907 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e473c3-08a9-4e62-83b5-e8c60aa2dbfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:47.201877 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e473c3-08a9-4e62-83b5-e8c60aa2dbfd None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:47.202229 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e473c3-08a9-4e62-83b5-e8c60aa2dbfd None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:47.202573 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e473c3-08a9-4e62-83b5-e8c60aa2dbfd None admin] PATH_INFO: `/v3/services/compute` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:47.203194 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e473c3-08a9-4e62-83b5-e8c60aa2dbfd None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:47.204566 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e473c3-08a9-4e62-83b5-e8c60aa2dbfd None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:47.213856 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-72e473c3-08a9-4e62-83b5-e8c60aa2dbfd None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 12:00:47.213856 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:47.213856 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 12:00:47.213856 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 12:00:47.213856 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 12:00:47.213856 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 12:00:47.213856 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 12:00:47.213856 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 12:00:47.213856 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 12:00:47.213856 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:00:47.213856 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 12:00:47.213856 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:00:47.213856 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:47.213856 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:47.213856 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:47.213856 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:47.213856 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:47.214882 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:47.214882 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:47.214882 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:47.214882 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:47.214882 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:47.214882 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:47.214882 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 12:00:47.214882 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 12:00:47.214882 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 12:00:47.214882 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 12:00:47.214882 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:47.214882 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:47.214882 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:47.214882 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:47.214882 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:47.214882 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:47.215756 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:47.215756 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:47.215756 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:47.215756 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:47.215756 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:47.215756 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:47.215756 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:47.215756 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:47.215756 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:47.215756 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:47.215756 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 12:00:47.215756 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 12:00:47.215756 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 12:00:47.215756 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:00:47.215756 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 479/961] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 9 12:00:47 2022] GET /identity/v3/services/compute => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:47.217129 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e473c3-08a9-4e62-83b5-e8c60aa2dbfd None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:47.242090 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1001b7-6dc1-47c6-ad79-0c4790cf9c7b None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:47.280469 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba607f2-85af-48a7-8c53-29f720db610d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:47.281079 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba607f2-85af-48a7-8c53-29f720db610d None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:47.281365 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba607f2-85af-48a7-8c53-29f720db610d None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:47.281636 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba607f2-85af-48a7-8c53-29f720db610d None admin] PATH_INFO: `/v3/services` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:47.282303 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba607f2-85af-48a7-8c53-29f720db610d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:47.282607 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba607f2-85af-48a7-8c53-29f720db610d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:47.284559 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba607f2-85af-48a7-8c53-29f720db610d None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:47.290003 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 483/962] 10.222.0.49 () {56 vars in 1122 bytes} [Fri Sep 9 12:00:47 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:47.290782 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba607f2-85af-48a7-8c53-29f720db610d None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:47.327584 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e473c3-08a9-4e62-83b5-e8c60aa2dbfd None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:47.336608 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:47.343705 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:47.365412 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d5c948-15ee-4e89-bec5-3802dcd95174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:47.366047 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d5c948-15ee-4e89-bec5-3802dcd95174 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:47.366324 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d5c948-15ee-4e89-bec5-3802dcd95174 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:47.366592 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d5c948-15ee-4e89-bec5-3802dcd95174 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:47.374595 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d5c948-15ee-4e89-bec5-3802dcd95174 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:47.376170 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d5c948-15ee-4e89-bec5-3802dcd95174 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:47.388719 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-c7d5c948-15ee-4e89-bec5-3802dcd95174 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 12:00:47.388719 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:47.388719 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:47.388719 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:47.388719 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:47.388719 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:47.388719 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:47.388719 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:47.388719 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:47.388719 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:47.388719 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:47.388719 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:47.388719 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 12:00:47.388719 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 12:00:47.388719 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 12:00:47.388719 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 12:00:47.389669 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:47.389669 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:47.389669 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 12:00:47.389669 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:00:47.389669 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 12:00:47.389669 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:00:47.389669 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:47.389669 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:47.389669 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:47.389669 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:47.389669 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:47.389669 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:47.389669 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:47.389669 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:47.389669 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:47.389669 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:47.389669 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:47.389669 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:47.390609 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:47.390609 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:47.390609 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 12:00:47.390609 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 12:00:47.390609 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 12:00:47.390609 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 12:00:47.390609 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 12:00:47.390609 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 12:00:47.390609 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 12:00:47.390609 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:00:47.390609 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 480/963] 10.222.0.49 () {56 vars in 1117 bytes} [Fri Sep 9 12:00:47 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:47.390609 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d5c948-15ee-4e89-bec5-3802dcd95174 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:47.404417 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba607f2-85af-48a7-8c53-29f720db610d None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:47.441077 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d7e686-5aed-4f54-8549-c7d62e931c7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:47.441719 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d7e686-5aed-4f54-8549-c7d62e931c7f None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:47.441998 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d7e686-5aed-4f54-8549-c7d62e931c7f None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:47.442272 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d7e686-5aed-4f54-8549-c7d62e931c7f None admin] PATH_INFO: `/v3/services` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:47.442923 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d7e686-5aed-4f54-8549-c7d62e931c7f None admin] RBAC: Adding query filter params (name=compute) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:47.443236 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d7e686-5aed-4f54-8549-c7d62e931c7f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:47.444601 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d7e686-5aed-4f54-8549-c7d62e931c7f None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:47.449285 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 484/964] 10.222.0.49 () {56 vars in 1112 bytes} [Fri Sep 9 12:00:47 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:47.450066 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d7e686-5aed-4f54-8549-c7d62e931c7f None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:47.509872 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d5c948-15ee-4e89-bec5-3802dcd95174 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:47.547210 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cda372f-037a-4de7-8d3b-e479271badf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:47.547826 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cda372f-037a-4de7-8d3b-e479271badf3 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:47.548103 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cda372f-037a-4de7-8d3b-e479271badf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:47.548381 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cda372f-037a-4de7-8d3b-e479271badf3 None admin] PATH_INFO: `/v3/services` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:47.549043 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cda372f-037a-4de7-8d3b-e479271badf3 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:47.549886 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cda372f-037a-4de7-8d3b-e479271badf3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:47.551445 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cda372f-037a-4de7-8d3b-e479271badf3 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:47.556718 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 481/965] 10.222.0.49 () {56 vars in 1122 bytes} [Fri Sep 9 12:00:47 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:47.557354 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d7e686-5aed-4f54-8549-c7d62e931c7f None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:47.560585 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:47.562208 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cda372f-037a-4de7-8d3b-e479271badf3 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:47.595411 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37501ed-31f2-458f-b711-92ddb38b1ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:47.596642 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37501ed-31f2-458f-b711-92ddb38b1ad7 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:47.596920 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37501ed-31f2-458f-b711-92ddb38b1ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:47.597187 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37501ed-31f2-458f-b711-92ddb38b1ad7 None admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:47.597911 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37501ed-31f2-458f-b711-92ddb38b1ad7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:47.598212 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37501ed-31f2-458f-b711-92ddb38b1ad7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:47.599659 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37501ed-31f2-458f-b711-92ddb38b1ad7 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:47.607156 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:47.623246 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 485/966] 10.222.0.49 () {56 vars in 1149 bytes} [Fri Sep 9 12:00:47 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:47.624380 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37501ed-31f2-458f-b711-92ddb38b1ad7 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:47.681921 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cda372f-037a-4de7-8d3b-e479271badf3 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:47.717688 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ab541c-b9fd-4ac3-a58b-79be4af3405a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:47.718300 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ab541c-b9fd-4ac3-a58b-79be4af3405a None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:47.718578 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ab541c-b9fd-4ac3-a58b-79be4af3405a None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:47.718851 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ab541c-b9fd-4ac3-a58b-79be4af3405a None admin] PATH_INFO: `/v3/services` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:47.719509 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ab541c-b9fd-4ac3-a58b-79be4af3405a None admin] RBAC: Adding query filter params (type=compute) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:47.719811 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ab541c-b9fd-4ac3-a58b-79be4af3405a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:47.721224 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ab541c-b9fd-4ac3-a58b-79be4af3405a None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:47.726043 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 482/967] 10.222.0.49 () {56 vars in 1112 bytes} [Fri Sep 9 12:00:47 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:47.726837 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ab541c-b9fd-4ac3-a58b-79be4af3405a None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:47.743637 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37501ed-31f2-458f-b711-92ddb38b1ad7 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:47.787860 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba9f2fe-fc29-46f1-901b-4471644b9e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:47.788660 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba9f2fe-fc29-46f1-901b-4471644b9e25 None admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:47.789067 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba9f2fe-fc29-46f1-901b-4471644b9e25 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:47.789439 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba9f2fe-fc29-46f1-901b-4471644b9e25 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:47.790377 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba9f2fe-fc29-46f1-901b-4471644b9e25 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:47.792126 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba9f2fe-fc29-46f1-901b-4471644b9e25 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:47.818944 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 486/968] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:00:47 2022] POST /identity/v3/endpoints => generated 349 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:00:47.886934 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ab541c-b9fd-4ac3-a58b-79be4af3405a None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:47.889377 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:47.942495 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4a7ec4-7432-44e1-a41f-fac7e0636641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:47.943163 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4a7ec4-7432-44e1-a41f-fac7e0636641 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:47.943442 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4a7ec4-7432-44e1-a41f-fac7e0636641 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:47.943713 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4a7ec4-7432-44e1-a41f-fac7e0636641 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:48.024501 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4a7ec4-7432-44e1-a41f-fac7e0636641 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:48.025965 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4a7ec4-7432-44e1-a41f-fac7e0636641 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:48.066588 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0a03fe-e590-49dd-8ffc-70b9e86af4b3 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:48.067221 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0a03fe-e590-49dd-8ffc-70b9e86af4b3 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:48.067677 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0a03fe-e590-49dd-8ffc-70b9e86af4b3 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:48.069706 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 487/969] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:00:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:00:48.076258 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f6b958-8856-4fa9-9a46-896248c7ae93 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:48.076750 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f6b958-8856-4fa9-9a46-896248c7ae93 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:48.077249 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f6b958-8856-4fa9-9a46-896248c7ae93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:48.100641 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-b1f6b958-8856-4fa9-9a46-896248c7ae93 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:00:48.101844 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-b1f6b958-8856-4fa9-9a46-896248c7ae93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:48.122712 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:48.136218 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:48.141828 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 483/970] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 9 12:00:47 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 12:00:48.149982 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4a7ec4-7432-44e1-a41f-fac7e0636641 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:48.192117 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 488/971] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:00:48 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 12:00:48.264025 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4a7ec4-7432-44e1-a41f-fac7e0636641 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:48.300681 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f6a03a-d766-4235-b2d5-58f95c7823ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:48.301371 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f6a03a-d766-4235-b2d5-58f95c7823ae None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:48.301652 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f6a03a-d766-4235-b2d5-58f95c7823ae None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:48.301922 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f6a03a-d766-4235-b2d5-58f95c7823ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:48.312420 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f6a03a-d766-4235-b2d5-58f95c7823ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:48.313864 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f6a03a-d766-4235-b2d5-58f95c7823ae None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:48.328906 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-69f6a03a-d766-4235-b2d5-58f95c7823ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:00:48.328906 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:48.328906 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:48.328906 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:48.328906 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:48.328906 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:48.328906 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:48.328906 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:48.328906 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:48.328906 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:48.328906 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:48.328906 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:48.328906 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:00:48.328906 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:00:48.328906 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:00:48.328906 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:00:48.329844 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:48.329844 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:48.329844 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:48.329844 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:48.329844 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:48.329844 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:48.329844 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:48.329844 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:48.329844 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:48.329844 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:48.329844 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:48.329844 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:48.329844 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:48.329844 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:48.329844 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:48.329844 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:48.329844 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:00:48.330856 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:00:48.330856 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:00:48.330856 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:00:48.330856 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 484/972] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 12:00:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 12:00:48.336065 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f6b958-8856-4fa9-9a46-896248c7ae93 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:48.445142 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f6b958-8856-4fa9-9a46-896248c7ae93 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:48.491139 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ac4eef-79e7-4cd0-8080-a491e18a77a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:48.491754 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ac4eef-79e7-4cd0-8080-a491e18a77a4 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:48.492034 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ac4eef-79e7-4cd0-8080-a491e18a77a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:48.492307 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ac4eef-79e7-4cd0-8080-a491e18a77a4 None admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:48.492951 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ac4eef-79e7-4cd0-8080-a491e18a77a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:48.493253 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ac4eef-79e7-4cd0-8080-a491e18a77a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:48.494624 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ac4eef-79e7-4cd0-8080-a491e18a77a4 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:48.503923 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f6a03a-d766-4235-b2d5-58f95c7823ae None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:48.507060 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 489/973] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 9 12:00:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:48.512344 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ac4eef-79e7-4cd0-8080-a491e18a77a4 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:48.525531 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:48.570924 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:48.638521 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ac4eef-79e7-4cd0-8080-a491e18a77a4 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:48.641382 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f6a03a-d766-4235-b2d5-58f95c7823ae None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:48.682585 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6d37d8-e1cd-4035-b0db-ab630671c03c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:48.683297 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6d37d8-e1cd-4035-b0db-ab630671c03c None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:48.683574 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6d37d8-e1cd-4035-b0db-ab630671c03c None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:48.683844 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6d37d8-e1cd-4035-b0db-ab630671c03c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:48.691996 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4d481c-53c6-422e-892a-f256eb053454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:48.692920 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4d481c-53c6-422e-892a-f256eb053454 None admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:48.693181 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6d37d8-e1cd-4035-b0db-ab630671c03c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:48.693599 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4d481c-53c6-422e-892a-f256eb053454 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:48.694128 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4d481c-53c6-422e-892a-f256eb053454 None admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:48.694643 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6d37d8-e1cd-4035-b0db-ab630671c03c None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:48.695397 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4d481c-53c6-422e-892a-f256eb053454 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:48.697118 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4d481c-53c6-422e-892a-f256eb053454 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:48.706558 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-8a6d37d8-e1cd-4035-b0db-ab630671c03c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:00:48.706558 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:48.706558 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:48.706558 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:48.706558 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:48.706558 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:48.706558 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:48.706558 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:48.706558 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:48.706558 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:48.706558 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:48.706558 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:48.706558 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 12:00:48.706558 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 12:00:48.706558 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 12:00:48.707511 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 12:00:48.707511 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:48.707511 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:48.707511 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:48.707511 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:48.707511 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:48.707511 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:48.707511 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:48.707511 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:48.707511 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:48.707511 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:48.707511 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:48.707511 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:48.707511 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:48.707511 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:48.707511 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:48.707511 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:48.708416 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 12:00:48.708416 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 12:00:48.708416 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 12:00:48.708416 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 12:00:48.708416 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 12:00:48.708416 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 12:00:48.708416 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:00:48.708416 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:00:48.712603 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 490/974] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 9 12:00:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:48.713594 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6d37d8-e1cd-4035-b0db-ab630671c03c None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:48.717010 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.sql.core [None req-0e4d481c-53c6-422e-892a-f256eb053454 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 09 12:00:48.717010 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 09 12:00:48.717010 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [parameters: {'user_id': '788383dace6c42f5ac21d0cac671c547', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 09 12:00:48.717010 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81590) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 09 12:00:48.726813 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-0e4d481c-53c6-422e-892a-f256eb053454 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 09 12:00:48.726813 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:48.726813 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 09 12:00:48.726813 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 09 12:00:48.726813 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 09 12:00:48.726813 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 09 12:00:48.726813 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 09 12:00:48.726813 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application result = self._query(query) Sep 09 12:00:48.726813 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 09 12:00:48.726813 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application conn.query(q) Sep 09 12:00:48.726813 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 09 12:00:48.726813 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 09 12:00:48.726813 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 09 12:00:48.726813 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application result.read() Sep 09 12:00:48.726813 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 09 12:00:48.727758 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 09 12:00:48.727758 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 09 12:00:48.727758 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 09 12:00:48.727758 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 09 12:00:48.727758 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 09 12:00:48.727758 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 09 12:00:48.727758 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 09 12:00:48.727758 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 09 12:00:48.727758 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:00:48.727758 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 09 12:00:48.727758 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:00:48.727758 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:48.727758 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 09 12:00:48.727758 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 09 12:00:48.727758 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 09 12:00:48.727758 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 09 12:00:48.727758 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 09 12:00:48.727758 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application next(self.gen) Sep 09 12:00:48.728699 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 09 12:00:48.728699 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application yield resource Sep 09 12:00:48.728699 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 09 12:00:48.728699 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application next(self.gen) Sep 09 12:00:48.728699 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 09 12:00:48.728699 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application self.session.rollback() Sep 09 12:00:48.728699 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 09 12:00:48.728699 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application self.force_reraise() Sep 09 12:00:48.728699 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 09 12:00:48.728699 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise self.value Sep 09 12:00:48.728699 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 09 12:00:48.728699 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 09 12:00:48.728699 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 09 12:00:48.728699 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application session.commit() Sep 09 12:00:48.728699 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 09 12:00:48.728699 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 09 12:00:48.728699 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 09 12:00:48.729614 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application self._prepare_impl() Sep 09 12:00:48.729614 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 09 12:00:48.729614 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application self.session.flush() Sep 09 12:00:48.729614 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 09 12:00:48.729614 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application self._flush(objects) Sep 09 12:00:48.729614 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 09 12:00:48.729614 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 09 12:00:48.729614 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 09 12:00:48.729614 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application compat.raise_( Sep 09 12:00:48.729614 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 09 12:00:48.729614 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception Sep 09 12:00:48.729614 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 09 12:00:48.729614 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application flush_context.execute() Sep 09 12:00:48.729614 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 09 12:00:48.729614 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rec.execute(self) Sep 09 12:00:48.729614 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 09 12:00:48.729614 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 09 12:00:48.730521 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 09 12:00:48.730521 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 09 12:00:48.730521 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 09 12:00:48.730521 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 09 12:00:48.730521 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 09 12:00:48.730521 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 09 12:00:48.730521 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 09 12:00:48.730521 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 09 12:00:48.730521 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 09 12:00:48.730521 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 09 12:00:48.730521 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 09 12:00:48.730521 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 09 12:00:48.730521 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 09 12:00:48.730521 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 09 12:00:48.730521 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 09 12:00:48.730521 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception Sep 09 12:00:48.731403 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 09 12:00:48.731403 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 09 12:00:48.731403 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 09 12:00:48.731403 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 09 12:00:48.731403 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 09 12:00:48.731403 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application result = self._query(query) Sep 09 12:00:48.731403 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 09 12:00:48.731403 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application conn.query(q) Sep 09 12:00:48.731403 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 09 12:00:48.731403 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 09 12:00:48.731403 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 09 12:00:48.731403 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application result.read() Sep 09 12:00:48.731403 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 09 12:00:48.731403 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 09 12:00:48.731403 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 09 12:00:48.731403 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 09 12:00:48.732251 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 09 12:00:48.732251 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 09 12:00:48.732251 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 09 12:00:48.732251 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 09 12:00:48.732251 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 09 12:00:48.732251 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 09 12:00:48.732251 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [parameters: {'user_id': '788383dace6c42f5ac21d0cac671c547', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 09 12:00:48.732251 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 09 12:00:48.732251 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:00:48.732251 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 12:00:48.732251 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:00:48.732251 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:48.732251 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:48.732251 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:48.732251 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:48.733032 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:48.733032 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:48.733032 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:48.733032 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:48.733032 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:48.733032 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:48.733032 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:48.733032 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 09 12:00:48.733032 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 09 12:00:48.733032 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:48.733032 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:48.733032 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 12:00:48.733032 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:00:48.733032 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 12:00:48.733032 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:00:48.733032 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 09 12:00:48.733032 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 09 12:00:48.734042 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 09 12:00:48.734042 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 09 12:00:48.734042 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 09 12:00:48.734042 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 09 12:00:48.734042 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 09 12:00:48.734042 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:00:48.743364 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 485/975] 10.222.0.49 () {60 vars in 1131 bytes} [Fri Sep 9 12:00:48 2022] POST /identity/v3/users => generated 160 bytes in 235 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 09 12:00:48.744777 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4d481c-53c6-422e-892a-f256eb053454 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:48.787479 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:48.829130 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6d37d8-e1cd-4035-b0db-ab630671c03c None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:48.866496 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4d481c-53c6-422e-892a-f256eb053454 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:48.868004 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0935fd14-b643-4d69-80e6-1525c315a8c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:48.868615 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0935fd14-b643-4d69-80e6-1525c315a8c3 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:48.868899 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0935fd14-b643-4d69-80e6-1525c315a8c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:48.869237 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0935fd14-b643-4d69-80e6-1525c315a8c3 None admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:48.869852 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0935fd14-b643-4d69-80e6-1525c315a8c3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:48.870162 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0935fd14-b643-4d69-80e6-1525c315a8c3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:48.871754 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0935fd14-b643-4d69-80e6-1525c315a8c3 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:48.883733 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:48.887050 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 491/976] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Sep 9 12:00:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:48.892090 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0935fd14-b643-4d69-80e6-1525c315a8c3 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:48.915777 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68797c35-01df-4e63-81af-a81081ad9350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:48.916852 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68797c35-01df-4e63-81af-a81081ad9350 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:48.917448 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68797c35-01df-4e63-81af-a81081ad9350 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:48.917934 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68797c35-01df-4e63-81af-a81081ad9350 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:48.928165 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68797c35-01df-4e63-81af-a81081ad9350 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:48.930147 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68797c35-01df-4e63-81af-a81081ad9350 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:48.943131 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-68797c35-01df-4e63-81af-a81081ad9350 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 12:00:48.943131 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:48.943131 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:48.943131 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:48.943131 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:48.943131 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:48.943131 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:48.943131 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:48.943131 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:48.943131 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:48.943131 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:48.943131 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:48.943131 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 12:00:48.943131 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 12:00:48.943131 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 12:00:48.943131 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 12:00:48.944094 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:48.944094 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:48.944094 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 12:00:48.944094 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:00:48.944094 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 12:00:48.944094 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:00:48.944094 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:48.944094 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:48.944094 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:48.944094 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:48.944094 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:48.944094 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:48.944094 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:48.944094 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:48.944094 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:48.944094 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:48.944094 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:48.944094 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:48.945061 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:48.945061 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:48.945061 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 12:00:48.945061 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 12:00:48.945061 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 12:00:48.945061 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 12:00:48.945061 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 12:00:48.945061 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 12:00:48.945061 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 12:00:48.945061 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:00:48.952407 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 486/977] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 9 12:00:48 2022] GET /identity/v3/users/cinder => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:48.953584 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68797c35-01df-4e63-81af-a81081ad9350 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:49.001618 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0935fd14-b643-4d69-80e6-1525c315a8c3 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:49.030894 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:49.041007 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf87c009-5eb8-4988-af2e-56adf43a5c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:49.042060 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf87c009-5eb8-4988-af2e-56adf43a5c6b None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:49.042356 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf87c009-5eb8-4988-af2e-56adf43a5c6b None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:49.042623 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf87c009-5eb8-4988-af2e-56adf43a5c6b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:49.043382 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf87c009-5eb8-4988-af2e-56adf43a5c6b None admin] RBAC: Adding query filter params (role.id=db537712f45d4e6980592d3ddefb312a, scope.project.id=fe863fcd75f1406ebbffc5016078f73e, user.id=e368b4e668c54e25bff80a8e8a8c3588) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:49.043692 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf87c009-5eb8-4988-af2e-56adf43a5c6b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:49.045487 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf87c009-5eb8-4988-af2e-56adf43a5c6b None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:49.057756 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 492/978] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Sep 9 12:00:48 2022] GET /identity/v3/role_assignments?user.id=e368b4e668c54e25bff80a8e8a8c3588&scope.project.id=fe863fcd75f1406ebbffc5016078f73e&role.id=db537712f45d4e6980592d3ddefb312a => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:49.083412 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68797c35-01df-4e63-81af-a81081ad9350 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:49.122638 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df59a1c8-cfe2-4255-86fe-b4b874e7ee8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:49.123559 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df59a1c8-cfe2-4255-86fe-b4b874e7ee8a None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:49.124025 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df59a1c8-cfe2-4255-86fe-b4b874e7ee8a None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:49.124442 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df59a1c8-cfe2-4255-86fe-b4b874e7ee8a None admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:49.125541 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df59a1c8-cfe2-4255-86fe-b4b874e7ee8a None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:49.126035 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df59a1c8-cfe2-4255-86fe-b4b874e7ee8a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:49.128504 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df59a1c8-cfe2-4255-86fe-b4b874e7ee8a None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:49.172225 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 487/979] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Sep 9 12:00:48 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:49.998621 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cb66bf-1d29-4be4-b16b-f0ef62c6f1ca None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:49.999148 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cb66bf-1d29-4be4-b16b-f0ef62c6f1ca None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:49.999148 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cb66bf-1d29-4be4-b16b-f0ef62c6f1ca None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:50.000466 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 493/980] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:00:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:00:50.007535 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f1d1bc-24fd-4d1d-bbf2-d40283e4265e None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:50.007818 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f1d1bc-24fd-4d1d-bbf2-d40283e4265e None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:50.008091 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f1d1bc-24fd-4d1d-bbf2-d40283e4265e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:50.026644 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-93f1d1bc-24fd-4d1d-bbf2-d40283e4265e None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:00:50.028187 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-93f1d1bc-24fd-4d1d-bbf2-d40283e4265e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:50.056993 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:50.106151 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 488/981] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:00:50 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 12:00:50.160095 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d267178b-b94d-444e-bf8d-9eaadcf6e15f None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:50.160391 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d267178b-b94d-444e-bf8d-9eaadcf6e15f None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:50.160650 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d267178b-b94d-444e-bf8d-9eaadcf6e15f None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:50.162121 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 494/982] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:00:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 12:00:50.167131 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede372d6-a15a-41f1-b6dc-f651ce02e1cd None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:50.167420 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede372d6-a15a-41f1-b6dc-f651ce02e1cd None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:50.167692 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede372d6-a15a-41f1-b6dc-f651ce02e1cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:50.186604 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-ede372d6-a15a-41f1-b6dc-f651ce02e1cd None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:00:50.187817 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-ede372d6-a15a-41f1-b6dc-f651ce02e1cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:50.256044 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 489/983] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:00:50 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 12:00:50.382416 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d267178b-b94d-444e-bf8d-9eaadcf6e15f None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:50.483410 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d267178b-b94d-444e-bf8d-9eaadcf6e15f None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:50.519745 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037c397f-5741-4c0d-8776-3842a8edcb76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:50.520444 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037c397f-5741-4c0d-8776-3842a8edcb76 None admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:50.520859 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037c397f-5741-4c0d-8776-3842a8edcb76 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:50.521364 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037c397f-5741-4c0d-8776-3842a8edcb76 None admin] PATH_INFO: `/v3/services` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:50.522085 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037c397f-5741-4c0d-8776-3842a8edcb76 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:50.522607 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede372d6-a15a-41f1-b6dc-f651ce02e1cd None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:50.523747 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037c397f-5741-4c0d-8776-3842a8edcb76 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:50.543070 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 495/984] 10.222.0.49 () {60 vars in 1141 bytes} [Fri Sep 9 12:00:50 2022] POST /identity/v3/services => generated 242 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:00:50.621649 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:50.638594 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede372d6-a15a-41f1-b6dc-f651ce02e1cd None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:50.687805 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7179957-5d21-4c7e-b5cd-ad1dcc79d3bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:50.688442 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7179957-5d21-4c7e-b5cd-ad1dcc79d3bf None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:50.688713 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7179957-5d21-4c7e-b5cd-ad1dcc79d3bf None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:50.688977 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7179957-5d21-4c7e-b5cd-ad1dcc79d3bf None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:50.690446 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7179957-5d21-4c7e-b5cd-ad1dcc79d3bf None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:50.691865 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7179957-5d21-4c7e-b5cd-ad1dcc79d3bf None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:50.701514 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-f7179957-5d21-4c7e-b5cd-ad1dcc79d3bf None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 12:00:50.701514 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:50.701514 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 12:00:50.701514 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 12:00:50.701514 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 12:00:50.701514 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 12:00:50.701514 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 12:00:50.701514 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 12:00:50.701514 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 12:00:50.701514 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:00:50.701514 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 12:00:50.701514 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:00:50.701514 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:50.701514 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:50.701514 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:50.701514 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:50.701514 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:50.702568 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:50.702568 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:50.702568 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:50.702568 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:50.702568 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:50.702568 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:50.702568 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 12:00:50.702568 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 12:00:50.702568 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 12:00:50.702568 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 12:00:50.702568 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:50.702568 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:50.702568 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:50.702568 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:50.702568 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:50.702568 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:50.707965 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:50.707965 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:50.707965 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:50.707965 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:50.707965 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:50.707965 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:50.707965 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:50.707965 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:50.707965 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:50.707965 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:50.707965 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 12:00:50.707965 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 12:00:50.707965 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 12:00:50.707965 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:00:50.707965 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 490/985] 10.222.0.49 () {56 vars in 1126 bytes} [Fri Sep 9 12:00:50 2022] GET /identity/v3/services/object-store => generated 93 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 12:00:50.709944 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037c397f-5741-4c0d-8776-3842a8edcb76 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:50.768372 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:50.819693 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037c397f-5741-4c0d-8776-3842a8edcb76 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:50.855447 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bbd211-8a81-4f04-a305-ccec0db7722a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:50.856058 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bbd211-8a81-4f04-a305-ccec0db7722a None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:50.856336 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bbd211-8a81-4f04-a305-ccec0db7722a None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:50.856605 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bbd211-8a81-4f04-a305-ccec0db7722a None admin] PATH_INFO: `/v3/services` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:50.857228 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bbd211-8a81-4f04-a305-ccec0db7722a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:50.857526 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bbd211-8a81-4f04-a305-ccec0db7722a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:50.859005 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bbd211-8a81-4f04-a305-ccec0db7722a None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:50.864864 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 496/986] 10.222.0.49 () {56 vars in 1122 bytes} [Fri Sep 9 12:00:50 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:50.870557 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7179957-5d21-4c7e-b5cd-ad1dcc79d3bf None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:50.979692 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7179957-5d21-4c7e-b5cd-ad1dcc79d3bf None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:51.016201 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925a39fe-e9ea-476c-a7ca-de2198c66d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:51.016814 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925a39fe-e9ea-476c-a7ca-de2198c66d7a None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:51.017091 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925a39fe-e9ea-476c-a7ca-de2198c66d7a None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:51.017359 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925a39fe-e9ea-476c-a7ca-de2198c66d7a None admin] PATH_INFO: `/v3/services` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:51.018034 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925a39fe-e9ea-476c-a7ca-de2198c66d7a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:51.018337 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925a39fe-e9ea-476c-a7ca-de2198c66d7a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:51.019715 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925a39fe-e9ea-476c-a7ca-de2198c66d7a None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:51.025064 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 491/987] 10.222.0.49 () {56 vars in 1122 bytes} [Fri Sep 9 12:00:50 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:51.030997 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bbd211-8a81-4f04-a305-ccec0db7722a None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:51.136350 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bbd211-8a81-4f04-a305-ccec0db7722a None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:51.152576 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:51.171726 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faad910-2027-4054-971e-2c8aa200787f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:51.172328 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faad910-2027-4054-971e-2c8aa200787f None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:51.172602 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faad910-2027-4054-971e-2c8aa200787f None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:51.172868 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faad910-2027-4054-971e-2c8aa200787f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:51.173559 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5faad910-2027-4054-971e-2c8aa200787f None admin] RBAC: Adding query filter params (interface=admin, service_id=526d33a9603a4b869299c9b155a834e1, region_id=RegionOne) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:51.174576 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5faad910-2027-4054-971e-2c8aa200787f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:51.175965 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5faad910-2027-4054-971e-2c8aa200787f None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:51.181077 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 497/988] 10.222.0.49 () {56 vars in 1249 bytes} [Fri Sep 9 12:00:51 2022] GET /identity/v3/endpoints?service_id=526d33a9603a4b869299c9b155a834e1&interface=admin®ion_id=RegionOne => generated 193 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:51.186858 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925a39fe-e9ea-476c-a7ca-de2198c66d7a None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:51.204205 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:51.302118 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925a39fe-e9ea-476c-a7ca-de2198c66d7a None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:51.337026 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68cab19-c64d-40d8-884a-0c4e2005f271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:51.337640 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68cab19-c64d-40d8-884a-0c4e2005f271 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:51.337919 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68cab19-c64d-40d8-884a-0c4e2005f271 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:51.338224 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68cab19-c64d-40d8-884a-0c4e2005f271 None admin] PATH_INFO: `/v3/services` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:51.338828 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68cab19-c64d-40d8-884a-0c4e2005f271 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:51.340222 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68cab19-c64d-40d8-884a-0c4e2005f271 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:51.346309 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcea6dc-bdd9-4a41-86da-41f23b77e11a None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:51.346624 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcea6dc-bdd9-4a41-86da-41f23b77e11a None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:51.346868 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcea6dc-bdd9-4a41-86da-41f23b77e11a None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:51.348412 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 498/989] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:00:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 12:00:51.348734 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 492/990] 10.222.0.49 () {56 vars in 1087 bytes} [Fri Sep 9 12:00:51 2022] GET /identity/v3/services => generated 1727 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:00:51.353892 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83c2b7a-2ddb-4ab9-a667-53d50cc7873c None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:51.354193 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83c2b7a-2ddb-4ab9-a667-53d50cc7873c None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:51.354454 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83c2b7a-2ddb-4ab9-a667-53d50cc7873c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:51.373980 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-b83c2b7a-2ddb-4ab9-a667-53d50cc7873c None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:00:51.375327 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-b83c2b7a-2ddb-4ab9-a667-53d50cc7873c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:51.455256 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 499/991] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:00:51 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 12:00:51.726534 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68cab19-c64d-40d8-884a-0c4e2005f271 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:51.807973 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:51.832824 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68cab19-c64d-40d8-884a-0c4e2005f271 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:51.867330 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12b2fce-e6c2-4346-afde-aa7ae7c9aabb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:51.867961 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12b2fce-e6c2-4346-afde-aa7ae7c9aabb None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:51.868236 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12b2fce-e6c2-4346-afde-aa7ae7c9aabb None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:51.868504 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12b2fce-e6c2-4346-afde-aa7ae7c9aabb None admin] PATH_INFO: `/v3/services/image` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:51.869081 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12b2fce-e6c2-4346-afde-aa7ae7c9aabb None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:51.870441 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12b2fce-e6c2-4346-afde-aa7ae7c9aabb None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:51.879284 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-b12b2fce-e6c2-4346-afde-aa7ae7c9aabb None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 12:00:51.879284 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:51.879284 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 12:00:51.879284 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 12:00:51.879284 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 12:00:51.879284 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 12:00:51.879284 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 12:00:51.879284 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 12:00:51.879284 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 12:00:51.879284 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:00:51.879284 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 12:00:51.879284 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:00:51.879284 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:51.879284 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:51.879284 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:51.879284 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:51.879284 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:51.880409 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:51.880409 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:51.880409 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:51.880409 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:51.880409 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:51.880409 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:51.880409 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 12:00:51.880409 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 12:00:51.880409 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 12:00:51.880409 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 12:00:51.880409 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:51.880409 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:51.880409 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:51.880409 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:51.880409 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:51.880409 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:51.881279 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:51.881279 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:51.881279 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:51.881279 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:51.881279 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:51.881279 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:51.881279 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:51.881279 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:51.881279 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:51.881279 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:51.881279 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 12:00:51.881279 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 12:00:51.881279 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 12:00:51.881279 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:00:51.881279 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 493/992] 10.222.0.49 () {56 vars in 1105 bytes} [Fri Sep 9 12:00:51 2022] GET /identity/v3/services/image => generated 86 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:51.886431 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83c2b7a-2ddb-4ab9-a667-53d50cc7873c None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:51.988931 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83c2b7a-2ddb-4ab9-a667-53d50cc7873c None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:51.999857 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:52.024514 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7aeb218-299d-4ec2-81ce-2e76bb2d903b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:52.025122 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aeb218-299d-4ec2-81ce-2e76bb2d903b None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:52.025401 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aeb218-299d-4ec2-81ce-2e76bb2d903b None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:52.025670 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aeb218-299d-4ec2-81ce-2e76bb2d903b None admin] PATH_INFO: `/v3/services` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:52.026290 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7aeb218-299d-4ec2-81ce-2e76bb2d903b None admin] RBAC: Adding query filter params (name=image) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:52.026592 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7aeb218-299d-4ec2-81ce-2e76bb2d903b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:52.027987 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7aeb218-299d-4ec2-81ce-2e76bb2d903b None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:52.038262 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 500/993] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 12:00:51 2022] GET /identity/v3/services?name=image => generated 122 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:52.040133 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12b2fce-e6c2-4346-afde-aa7ae7c9aabb None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:52.145973 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12b2fce-e6c2-4346-afde-aa7ae7c9aabb None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:52.181024 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94f7237-3982-4b53-9c67-fccbead4a723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:52.181631 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94f7237-3982-4b53-9c67-fccbead4a723 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:52.181912 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94f7237-3982-4b53-9c67-fccbead4a723 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:52.182191 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94f7237-3982-4b53-9c67-fccbead4a723 None admin] PATH_INFO: `/v3/services` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:52.182805 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94f7237-3982-4b53-9c67-fccbead4a723 None admin] RBAC: Adding query filter params (type=image) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:52.183135 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94f7237-3982-4b53-9c67-fccbead4a723 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:52.184510 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94f7237-3982-4b53-9c67-fccbead4a723 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:52.194943 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 494/994] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 12:00:52 2022] GET /identity/v3/services?type=image => generated 122 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:52.761970 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af5e651-8d68-496f-b65e-5629e230fe65 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:52.762329 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af5e651-8d68-496f-b65e-5629e230fe65 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:52.762514 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af5e651-8d68-496f-b65e-5629e230fe65 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:52.764064 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 501/995] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:00:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:00:52.769643 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e919d4b-062d-4456-b907-4e0791412963 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:52.769934 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e919d4b-062d-4456-b907-4e0791412963 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:52.770205 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e919d4b-062d-4456-b907-4e0791412963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:52.782741 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:52.788697 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-4e919d4b-062d-4456-b907-4e0791412963 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:00:52.790423 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-4e919d4b-062d-4456-b907-4e0791412963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:52.857088 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 495/996] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:00:52 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 12:00:53.127188 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af5e651-8d68-496f-b65e-5629e230fe65 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:53.226285 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af5e651-8d68-496f-b65e-5629e230fe65 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:53.259938 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e99508-bdf1-463c-b809-f8cea59003e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:53.260564 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e99508-bdf1-463c-b809-f8cea59003e3 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:53.260840 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e99508-bdf1-463c-b809-f8cea59003e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:53.261110 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e99508-bdf1-463c-b809-f8cea59003e3 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:53.261681 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e99508-bdf1-463c-b809-f8cea59003e3 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:53.263060 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e99508-bdf1-463c-b809-f8cea59003e3 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:53.272291 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-80e99508-bdf1-463c-b809-f8cea59003e3 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 12:00:53.272291 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:53.272291 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 12:00:53.272291 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 12:00:53.272291 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 12:00:53.272291 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 12:00:53.272291 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 12:00:53.272291 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 12:00:53.272291 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 12:00:53.272291 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:00:53.272291 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 12:00:53.272291 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:00:53.272291 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:53.272291 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:53.272291 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:53.272291 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:53.272291 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:53.273268 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:53.273268 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:53.273268 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:53.273268 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:53.273268 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:53.273268 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:53.273268 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 12:00:53.273268 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 12:00:53.273268 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 12:00:53.273268 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 12:00:53.273268 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:53.273268 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:53.273268 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:53.273268 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:53.273268 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:53.273268 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:53.274242 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:53.274242 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:53.274242 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:53.274242 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:53.274242 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:53.274242 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:53.274242 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:53.274242 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:53.274242 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:53.274242 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:53.274242 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 12:00:53.274242 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 12:00:53.274242 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 12:00:53.274242 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:00:53.281476 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 502/997] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 9 12:00:53 2022] GET /identity/v3/services/compute => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:53.281608 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e919d4b-062d-4456-b907-4e0791412963 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:53.382769 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e919d4b-062d-4456-b907-4e0791412963 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:53.418850 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f8faf5-7678-49dd-95b0-ace0c5c2d4fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:53.419761 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f8faf5-7678-49dd-95b0-ace0c5c2d4fd None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:53.420194 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f8faf5-7678-49dd-95b0-ace0c5c2d4fd None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:53.420613 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f8faf5-7678-49dd-95b0-ace0c5c2d4fd None admin] PATH_INFO: `/v3/services` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:53.421484 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f8faf5-7678-49dd-95b0-ace0c5c2d4fd None admin] RBAC: Adding query filter params (name=compute) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:53.422001 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f8faf5-7678-49dd-95b0-ace0c5c2d4fd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:53.423641 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f8faf5-7678-49dd-95b0-ace0c5c2d4fd None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:53.426099 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:53.435663 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 496/998] 10.222.0.49 () {56 vars in 1112 bytes} [Fri Sep 9 12:00:53 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:53.436826 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e99508-bdf1-463c-b809-f8cea59003e3 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:53.478591 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:53.537477 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6ebd82-0e47-400d-9d6d-8f75055d740e None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:53.537767 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6ebd82-0e47-400d-9d6d-8f75055d740e None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:53.538045 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6ebd82-0e47-400d-9d6d-8f75055d740e None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:53.542940 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 497/999] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:00:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 12:00:53.544477 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd90f865-b8ab-4468-b284-388f503be496 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:53.544759 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd90f865-b8ab-4468-b284-388f503be496 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:53.545031 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd90f865-b8ab-4468-b284-388f503be496 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:53.546857 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e99508-bdf1-463c-b809-f8cea59003e3 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:53.562739 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-bd90f865-b8ab-4468-b284-388f503be496 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:00:53.563968 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-bd90f865-b8ab-4468-b284-388f503be496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:53.582936 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03183c99-e238-4f0a-a446-2991bbe7e9b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:53.583613 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03183c99-e238-4f0a-a446-2991bbe7e9b4 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:53.583890 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03183c99-e238-4f0a-a446-2991bbe7e9b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:53.584159 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03183c99-e238-4f0a-a446-2991bbe7e9b4 None admin] PATH_INFO: `/v3/services` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:53.584783 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03183c99-e238-4f0a-a446-2991bbe7e9b4 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:53.585080 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03183c99-e238-4f0a-a446-2991bbe7e9b4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:53.586440 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03183c99-e238-4f0a-a446-2991bbe7e9b4 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:53.597184 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 503/1000] 10.222.0.49 () {56 vars in 1112 bytes} [Fri Sep 9 12:00:53 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:53.598075 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03183c99-e238-4f0a-a446-2991bbe7e9b4 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:53.646111 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 498/1001] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:00:53 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 12:00:53.712787 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03183c99-e238-4f0a-a446-2991bbe7e9b4 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:53.714416 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:53.750931 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d406d21-a1ac-432f-9f2d-22cb59bd2cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:53.751547 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d406d21-a1ac-432f-9f2d-22cb59bd2cdc None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:53.752421 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d406d21-a1ac-432f-9f2d-22cb59bd2cdc None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:53.752697 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d406d21-a1ac-432f-9f2d-22cb59bd2cdc None admin] PATH_INFO: `/v3/endpoints` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:53.753491 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d406d21-a1ac-432f-9f2d-22cb59bd2cdc None admin] RBAC: Adding query filter params (interface=public, service_id=fd0e4d8cc72e4f99a6b8c10fcc84aa01, region_id=RegionOne) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:53.753908 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d406d21-a1ac-432f-9f2d-22cb59bd2cdc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:53.755309 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d406d21-a1ac-432f-9f2d-22cb59bd2cdc None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:53.764944 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 504/1002] 10.222.0.49 () {56 vars in 1251 bytes} [Fri Sep 9 12:00:53 2022] GET /identity/v3/endpoints?service_id=fd0e4d8cc72e4f99a6b8c10fcc84aa01&interface=public®ion_id=RegionOne => generated 194 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:00:53.766473 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd90f865-b8ab-4468-b284-388f503be496 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:53.868578 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd90f865-b8ab-4468-b284-388f503be496 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:53.897313 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:53.904265 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da10749-4b32-4a25-b5e0-c4020d011d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:53.904877 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da10749-4b32-4a25-b5e0-c4020d011d7c None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:53.905156 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da10749-4b32-4a25-b5e0-c4020d011d7c None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:53.905427 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da10749-4b32-4a25-b5e0-c4020d011d7c None admin] PATH_INFO: `/v3/services` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:53.906028 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da10749-4b32-4a25-b5e0-c4020d011d7c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:53.907444 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da10749-4b32-4a25-b5e0-c4020d011d7c None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:53.913270 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d406d21-a1ac-432f-9f2d-22cb59bd2cdc None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:53.919161 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 499/1003] 10.222.0.49 () {56 vars in 1087 bytes} [Fri Sep 9 12:00:53 2022] GET /identity/v3/services => generated 1727 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:00:54.028729 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d406d21-a1ac-432f-9f2d-22cb59bd2cdc None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:54.066051 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec7bded-c436-45de-b353-8db16b9c8231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:54.066699 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec7bded-c436-45de-b353-8db16b9c8231 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:54.067007 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec7bded-c436-45de-b353-8db16b9c8231 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:54.067285 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec7bded-c436-45de-b353-8db16b9c8231 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:54.067879 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec7bded-c436-45de-b353-8db16b9c8231 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:54.069280 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec7bded-c436-45de-b353-8db16b9c8231 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:54.079231 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-4ec7bded-c436-45de-b353-8db16b9c8231 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 12:00:54.079231 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:54.079231 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 12:00:54.079231 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 12:00:54.079231 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 12:00:54.079231 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 12:00:54.079231 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 12:00:54.079231 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 12:00:54.079231 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 12:00:54.079231 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:00:54.079231 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 12:00:54.079231 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:00:54.079231 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:54.079231 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:54.079231 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:54.079231 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:54.079231 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:54.080654 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:54.080654 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:54.080654 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:54.080654 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:54.080654 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:54.080654 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:54.080654 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 12:00:54.080654 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 12:00:54.080654 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 12:00:54.080654 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 12:00:54.080654 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:54.080654 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:54.080654 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:54.080654 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:54.080654 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:54.080654 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:54.081497 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:54.081497 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:54.081497 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:54.081497 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:54.081497 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:54.081497 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:54.081497 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:54.081497 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:54.081497 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:54.081497 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:54.081497 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 12:00:54.081497 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 12:00:54.081497 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 12:00:54.081497 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:00:54.086925 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 505/1004] 10.222.0.49 () {56 vars in 1126 bytes} [Fri Sep 9 12:00:53 2022] GET /identity/v3/services/object-store => generated 93 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 12:00:54.087073 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da10749-4b32-4a25-b5e0-c4020d011d7c None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:54.190642 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da10749-4b32-4a25-b5e0-c4020d011d7c None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:54.225334 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e436a4-061b-4871-ae09-1b468264f90b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:54.226190 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e436a4-061b-4871-ae09-1b468264f90b None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:54.226641 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e436a4-061b-4871-ae09-1b468264f90b None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:54.227102 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e436a4-061b-4871-ae09-1b468264f90b None admin] PATH_INFO: `/v3/services` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:54.227929 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e436a4-061b-4871-ae09-1b468264f90b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:54.228402 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e436a4-061b-4871-ae09-1b468264f90b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:54.229940 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e436a4-061b-4871-ae09-1b468264f90b None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:54.234876 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 500/1005] 10.222.0.49 () {56 vars in 1122 bytes} [Fri Sep 9 12:00:54 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:54.241359 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec7bded-c436-45de-b353-8db16b9c8231 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:54.272762 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:54.354094 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec7bded-c436-45de-b353-8db16b9c8231 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:54.388823 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f810ec8d-0026-4109-a7dd-8774172b6d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:54.389434 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f810ec8d-0026-4109-a7dd-8774172b6d78 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:54.389772 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f810ec8d-0026-4109-a7dd-8774172b6d78 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:54.390044 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f810ec8d-0026-4109-a7dd-8774172b6d78 None admin] PATH_INFO: `/v3/services` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:54.390729 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f810ec8d-0026-4109-a7dd-8774172b6d78 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:54.391069 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f810ec8d-0026-4109-a7dd-8774172b6d78 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:54.393058 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f810ec8d-0026-4109-a7dd-8774172b6d78 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:54.403081 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 506/1006] 10.222.0.49 () {56 vars in 1122 bytes} [Fri Sep 9 12:00:54 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:00:54.404017 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e436a4-061b-4871-ae09-1b468264f90b None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:54.412608 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161fe813-089a-4c36-9749-2c88649ac7fe None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:54.413036 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161fe813-089a-4c36-9749-2c88649ac7fe None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:54.413312 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161fe813-089a-4c36-9749-2c88649ac7fe None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:54.414793 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 507/1007] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:00:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:00:54.419730 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da14b6c7-d8b4-44ac-ade0-c37dde06e4ea None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:54.420012 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da14b6c7-d8b4-44ac-ade0-c37dde06e4ea None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:54.420285 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da14b6c7-d8b4-44ac-ade0-c37dde06e4ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:54.438860 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-da14b6c7-d8b4-44ac-ade0-c37dde06e4ea None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:00:54.441071 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-da14b6c7-d8b4-44ac-ade0-c37dde06e4ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:54.471988 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:54.511933 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 508/1008] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:00:54 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 12:00:54.517686 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e436a4-061b-4871-ae09-1b468264f90b None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:54.553190 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8087db5-5a45-4447-aea9-6244ebd5054a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:54.553897 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8087db5-5a45-4447-aea9-6244ebd5054a None admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:54.554239 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8087db5-5a45-4447-aea9-6244ebd5054a None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:54.554576 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8087db5-5a45-4447-aea9-6244ebd5054a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:54.555343 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8087db5-5a45-4447-aea9-6244ebd5054a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:54.556764 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8087db5-5a45-4447-aea9-6244ebd5054a None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:54.574941 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 501/1009] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:00:54 2022] POST /identity/v3/endpoints => generated 325 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:00:54.788639 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da14b6c7-d8b4-44ac-ade0-c37dde06e4ea None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:54.816509 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:54.894238 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da14b6c7-d8b4-44ac-ade0-c37dde06e4ea None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:54.933453 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a48804-3d4a-4878-b019-e025ac8cb217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:54.934063 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a48804-3d4a-4878-b019-e025ac8cb217 None admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:54.934343 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a48804-3d4a-4878-b019-e025ac8cb217 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:54.934612 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a48804-3d4a-4878-b019-e025ac8cb217 None admin] PATH_INFO: `/v3/services` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:54.935286 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a48804-3d4a-4878-b019-e025ac8cb217 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:54.936641 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a48804-3d4a-4878-b019-e025ac8cb217 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:54.953535 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 509/1010] 10.222.0.49 () {60 vars in 1141 bytes} [Fri Sep 9 12:00:54 2022] POST /identity/v3/services => generated 242 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:00:56.105711 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5424bba-915c-41c5-9885-81bb4c26a7fe None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:56.106301 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5424bba-915c-41c5-9885-81bb4c26a7fe None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:56.106301 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5424bba-915c-41c5-9885-81bb4c26a7fe None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:56.107748 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 502/1011] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:00:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:00:56.113472 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ea9ab7-9419-4f2d-91f7-c4b974bd049e None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:56.113852 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ea9ab7-9419-4f2d-91f7-c4b974bd049e None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:56.114027 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ea9ab7-9419-4f2d-91f7-c4b974bd049e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:56.132269 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-b3ea9ab7-9419-4f2d-91f7-c4b974bd049e None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:00:56.133432 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-b3ea9ab7-9419-4f2d-91f7-c4b974bd049e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:56.208119 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 510/1012] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:00:56 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 12:00:56.479803 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5424bba-915c-41c5-9885-81bb4c26a7fe None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:56.585791 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5424bba-915c-41c5-9885-81bb4c26a7fe None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:56.603189 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:56.621683 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef4fa40-18ff-44c0-8375-b55c1ed5b091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:56.622311 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef4fa40-18ff-44c0-8375-b55c1ed5b091 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:56.622585 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef4fa40-18ff-44c0-8375-b55c1ed5b091 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:56.622840 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef4fa40-18ff-44c0-8375-b55c1ed5b091 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:56.623440 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef4fa40-18ff-44c0-8375-b55c1ed5b091 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:56.624807 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef4fa40-18ff-44c0-8375-b55c1ed5b091 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:56.638716 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-5ef4fa40-18ff-44c0-8375-b55c1ed5b091 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 12:00:56.638716 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:56.638716 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 12:00:56.638716 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 12:00:56.638716 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 12:00:56.638716 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 12:00:56.638716 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 12:00:56.638716 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 12:00:56.638716 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 12:00:56.638716 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:00:56.638716 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 12:00:56.638716 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:00:56.638716 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:56.638716 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:56.638716 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:56.638716 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:56.638716 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:56.639825 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:56.639825 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:56.639825 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:56.639825 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:56.639825 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:56.639825 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:56.639825 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 12:00:56.639825 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 12:00:56.639825 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 12:00:56.639825 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 12:00:56.639825 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:56.639825 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:56.639825 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:56.639825 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:56.639825 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:56.639825 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:56.640720 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:56.640720 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:56.640720 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:56.640720 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:56.640720 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:56.640720 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:56.640720 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:56.640720 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:56.640720 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:56.640720 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:56.640720 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 12:00:56.640720 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 12:00:56.640720 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 12:00:56.640720 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:00:56.646414 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 503/1013] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 9 12:00:56 2022] GET /identity/v3/services/compute => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:56.647546 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ea9ab7-9419-4f2d-91f7-c4b974bd049e None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:56.656178 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:56.751884 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ea9ab7-9419-4f2d-91f7-c4b974bd049e None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:56.763750 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7317e0c1-477f-4fc4-b1c9-9d44472c1c74 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:56.764035 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7317e0c1-477f-4fc4-b1c9-9d44472c1c74 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:56.764302 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7317e0c1-477f-4fc4-b1c9-9d44472c1c74 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:56.765641 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 504/1014] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:00:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:00:56.771231 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa02c2d-9615-42e0-853b-d2d58c4e4373 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:56.771516 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa02c2d-9615-42e0-853b-d2d58c4e4373 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:56.771786 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa02c2d-9615-42e0-853b-d2d58c4e4373 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:56.789510 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aed6045-5cfa-466b-a1ba-f853b340cfa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:56.790113 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aed6045-5cfa-466b-a1ba-f853b340cfa5 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:56.790388 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aed6045-5cfa-466b-a1ba-f853b340cfa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:56.790656 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aed6045-5cfa-466b-a1ba-f853b340cfa5 None admin] PATH_INFO: `/v3/services` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:56.790823 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-7aa02c2d-9615-42e0-853b-d2d58c4e4373 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:00:56.791303 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aed6045-5cfa-466b-a1ba-f853b340cfa5 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:56.791598 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aed6045-5cfa-466b-a1ba-f853b340cfa5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:56.791999 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-7aa02c2d-9615-42e0-853b-d2d58c4e4373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:56.792948 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aed6045-5cfa-466b-a1ba-f853b340cfa5 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:56.797876 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 511/1015] 10.222.0.49 () {56 vars in 1112 bytes} [Fri Sep 9 12:00:56 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:00:56.803611 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aed6045-5cfa-466b-a1ba-f853b340cfa5 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:56.860633 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 505/1016] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:00:56 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 12:00:56.875197 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:56.906421 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aed6045-5cfa-466b-a1ba-f853b340cfa5 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:56.941146 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5505d9-aeea-4e84-ab7b-96a05e476b40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:56.941739 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5505d9-aeea-4e84-ab7b-96a05e476b40 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:56.942013 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5505d9-aeea-4e84-ab7b-96a05e476b40 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:56.942286 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5505d9-aeea-4e84-ab7b-96a05e476b40 None admin] PATH_INFO: `/v3/services` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:56.942922 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5505d9-aeea-4e84-ab7b-96a05e476b40 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:56.943220 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5505d9-aeea-4e84-ab7b-96a05e476b40 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:56.944563 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5505d9-aeea-4e84-ab7b-96a05e476b40 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:56.953941 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 512/1017] 10.222.0.49 () {56 vars in 1112 bytes} [Fri Sep 9 12:00:56 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:00:56.956149 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa02c2d-9615-42e0-853b-d2d58c4e4373 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:57.064191 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa02c2d-9615-42e0-853b-d2d58c4e4373 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:57.078085 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:57.102244 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3816a121-980c-438f-8f66-8c69697497d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:57.103055 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3816a121-980c-438f-8f66-8c69697497d4 None admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:57.103267 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3816a121-980c-438f-8f66-8c69697497d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:57.103527 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3816a121-980c-438f-8f66-8c69697497d4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:57.104187 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3816a121-980c-438f-8f66-8c69697497d4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:57.105550 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3816a121-980c-438f-8f66-8c69697497d4 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:57.127421 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 506/1018] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:00:56 2022] POST /identity/v3/endpoints => generated 334 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:00:57.134954 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5505d9-aeea-4e84-ab7b-96a05e476b40 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:57.142558 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d42164-9ae4-468e-bced-aa9ee1dea55f None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:57.142864 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d42164-9ae4-468e-bced-aa9ee1dea55f None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:57.143136 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d42164-9ae4-468e-bced-aa9ee1dea55f None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:57.144562 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 507/1019] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:00:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:00:57.150313 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8042d4b9-a3ad-4999-8ae5-47355b1ca3a6 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:57.150597 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8042d4b9-a3ad-4999-8ae5-47355b1ca3a6 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:57.150871 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8042d4b9-a3ad-4999-8ae5-47355b1ca3a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:57.170397 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-8042d4b9-a3ad-4999-8ae5-47355b1ca3a6 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:00:57.171659 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-8042d4b9-a3ad-4999-8ae5-47355b1ca3a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:57.254216 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5505d9-aeea-4e84-ab7b-96a05e476b40 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:57.258535 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 508/1020] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:00:57 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 12:00:57.292929 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90834b47-57eb-4585-8448-7bbf5d8e2cfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:57.293577 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90834b47-57eb-4585-8448-7bbf5d8e2cfc None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:57.293855 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90834b47-57eb-4585-8448-7bbf5d8e2cfc None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:57.294124 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90834b47-57eb-4585-8448-7bbf5d8e2cfc None admin] PATH_INFO: `/v3/domains/default` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:57.295301 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90834b47-57eb-4585-8448-7bbf5d8e2cfc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:57.296782 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90834b47-57eb-4585-8448-7bbf5d8e2cfc None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:57.299326 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 513/1021] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 12:00:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:57.305141 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8042d4b9-a3ad-4999-8ae5-47355b1ca3a6 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:57.403968 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:57.412735 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8042d4b9-a3ad-4999-8ae5-47355b1ca3a6 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:57.449872 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14076358-bcb1-4205-8b8f-72d36da24e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:57.450490 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14076358-bcb1-4205-8b8f-72d36da24e70 None admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:57.450768 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14076358-bcb1-4205-8b8f-72d36da24e70 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:57.451077 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14076358-bcb1-4205-8b8f-72d36da24e70 None admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:57.451735 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14076358-bcb1-4205-8b8f-72d36da24e70 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:57.453166 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14076358-bcb1-4205-8b8f-72d36da24e70 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:57.481376 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 509/1022] 10.222.0.49 () {60 vars in 1141 bytes} [Fri Sep 9 12:00:57 2022] POST /identity/v3/projects => generated 311 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:00:57.536214 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90834b47-57eb-4585-8448-7bbf5d8e2cfc None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:57.539197 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:57.659183 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90834b47-57eb-4585-8448-7bbf5d8e2cfc None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:57.696533 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb43b9c2-f1f0-415d-9e1b-abcc816e6b2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:57.697178 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb43b9c2-f1f0-415d-9e1b-abcc816e6b2c None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:57.697459 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb43b9c2-f1f0-415d-9e1b-abcc816e6b2c None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:57.697730 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb43b9c2-f1f0-415d-9e1b-abcc816e6b2c None admin] PATH_INFO: `/v3/services/image` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:57.698306 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb43b9c2-f1f0-415d-9e1b-abcc816e6b2c None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:57.699765 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb43b9c2-f1f0-415d-9e1b-abcc816e6b2c None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:57.709392 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-bb43b9c2-f1f0-415d-9e1b-abcc816e6b2c None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 12:00:57.709392 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:57.709392 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 12:00:57.709392 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 12:00:57.709392 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 12:00:57.709392 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 12:00:57.709392 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 12:00:57.709392 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 12:00:57.709392 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 12:00:57.709392 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:00:57.709392 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 12:00:57.709392 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:00:57.709392 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:57.709392 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:57.709392 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:57.709392 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:57.709392 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:57.710412 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:57.710412 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:57.710412 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:57.710412 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:57.710412 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:57.710412 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:57.710412 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 12:00:57.710412 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 12:00:57.710412 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 12:00:57.710412 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 12:00:57.710412 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:57.710412 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:57.710412 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:57.710412 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:57.710412 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:57.710412 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:57.711303 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:57.711303 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:57.711303 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:57.711303 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:57.711303 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:57.711303 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:57.711303 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:57.711303 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:57.711303 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:57.711303 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:57.711303 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 12:00:57.711303 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 12:00:57.711303 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 12:00:57.711303 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:00:57.711303 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 514/1023] 10.222.0.49 () {56 vars in 1105 bytes} [Fri Sep 9 12:00:57 2022] GET /identity/v3/services/image => generated 86 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:57.718205 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14076358-bcb1-4205-8b8f-72d36da24e70 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:57.825859 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14076358-bcb1-4205-8b8f-72d36da24e70 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:57.861241 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7b12cc-1642-4301-95cf-c272f519042b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:57.861849 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7b12cc-1642-4301-95cf-c272f519042b None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:57.862122 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7b12cc-1642-4301-95cf-c272f519042b None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:57.862390 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7b12cc-1642-4301-95cf-c272f519042b None admin] PATH_INFO: `/v3/services` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:57.863037 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7b12cc-1642-4301-95cf-c272f519042b None admin] RBAC: Adding query filter params (name=image) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:57.863338 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7b12cc-1642-4301-95cf-c272f519042b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:57.864700 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7b12cc-1642-4301-95cf-c272f519042b None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:57.870803 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 510/1024] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 12:00:57 2022] GET /identity/v3/services?name=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:57.875673 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb43b9c2-f1f0-415d-9e1b-abcc816e6b2c None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:57.945436 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:57.988315 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb43b9c2-f1f0-415d-9e1b-abcc816e6b2c None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:58.022422 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0e9185-c3fc-4f35-b025-0f5050055af0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:58.023188 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0e9185-c3fc-4f35-b025-0f5050055af0 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:58.023470 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0e9185-c3fc-4f35-b025-0f5050055af0 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:58.023744 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0e9185-c3fc-4f35-b025-0f5050055af0 None admin] PATH_INFO: `/v3/services` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:58.024396 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0e9185-c3fc-4f35-b025-0f5050055af0 None admin] RBAC: Adding query filter params (type=image) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:58.024696 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0e9185-c3fc-4f35-b025-0f5050055af0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:58.026054 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0e9185-c3fc-4f35-b025-0f5050055af0 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:58.031075 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 515/1025] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 12:00:57 2022] GET /identity/v3/services?type=image => generated 350 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:58.037698 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7b12cc-1642-4301-95cf-c272f519042b None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:58.051662 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:58.149483 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7b12cc-1642-4301-95cf-c272f519042b None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:58.184317 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a328772-a3e1-41d1-b033-d3ea9b15ec94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:58.184923 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a328772-a3e1-41d1-b033-d3ea9b15ec94 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:58.185201 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a328772-a3e1-41d1-b033-d3ea9b15ec94 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:58.185470 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a328772-a3e1-41d1-b033-d3ea9b15ec94 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:58.186160 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a328772-a3e1-41d1-b033-d3ea9b15ec94 None admin] RBAC: Adding query filter params (interface=public, service_id=d31b69b6264e4beab46e2c971395912c, region_id=RegionOne) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:00:58.186467 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a328772-a3e1-41d1-b033-d3ea9b15ec94 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:58.187906 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a328772-a3e1-41d1-b033-d3ea9b15ec94 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:58.198029 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 511/1026] 10.222.0.49 () {56 vars in 1251 bytes} [Fri Sep 9 12:00:58 2022] GET /identity/v3/endpoints?service_id=d31b69b6264e4beab46e2c971395912c&interface=public®ion_id=RegionOne => generated 194 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:00:58.199626 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0e9185-c3fc-4f35-b025-0f5050055af0 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:58.303567 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0e9185-c3fc-4f35-b025-0f5050055af0 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:58.331445 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:58.338463 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25c09c7-e05f-46e6-a99a-650766419862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:58.339093 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25c09c7-e05f-46e6-a99a-650766419862 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:58.339373 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25c09c7-e05f-46e6-a99a-650766419862 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:58.339643 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25c09c7-e05f-46e6-a99a-650766419862 None admin] PATH_INFO: `/v3/services` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:58.340241 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25c09c7-e05f-46e6-a99a-650766419862 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:00:58.341593 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25c09c7-e05f-46e6-a99a-650766419862 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:00:58.353598 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 516/1027] 10.222.0.49 () {56 vars in 1087 bytes} [Fri Sep 9 12:00:58 2022] GET /identity/v3/services => generated 1957 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:00:59.436334 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80023e19-1bd6-4a09-9ba1-693a1d56c7af None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:59.436856 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80023e19-1bd6-4a09-9ba1-693a1d56c7af None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:59.436856 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80023e19-1bd6-4a09-9ba1-693a1d56c7af None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:59.438395 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 512/1028] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:00:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:00:59.444198 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a2a0d1-c070-473d-89c5-5b6649d7fe00 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:59.444619 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a2a0d1-c070-473d-89c5-5b6649d7fe00 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:59.444797 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a2a0d1-c070-473d-89c5-5b6649d7fe00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:59.463558 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-f0a2a0d1-c070-473d-89c5-5b6649d7fe00 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:00:59.464752 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-f0a2a0d1-c070-473d-89c5-5b6649d7fe00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:59.530540 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 517/1029] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:00:59 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 12:00:59.705941 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba730f0-9935-43b9-a16d-e6e59307131f None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:59.706123 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba730f0-9935-43b9-a16d-e6e59307131f None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:59.706401 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba730f0-9935-43b9-a16d-e6e59307131f None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:59.707930 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 513/1030] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:00:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:00:59.712902 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2275358c-13a5-4f53-9d7f-c0795a6d9389 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:59.713190 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2275358c-13a5-4f53-9d7f-c0795a6d9389 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:59.713476 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2275358c-13a5-4f53-9d7f-c0795a6d9389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:59.731261 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-2275358c-13a5-4f53-9d7f-c0795a6d9389 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:00:59.732442 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-2275358c-13a5-4f53-9d7f-c0795a6d9389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:00:59.797405 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 518/1031] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:00:59 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 12:00:59.799428 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba730f0-9935-43b9-a16d-e6e59307131f None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:00:59.875682 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:00:59.903285 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba730f0-9935-43b9-a16d-e6e59307131f None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:00:59.937281 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd58a10-f3a4-42c8-ab7a-6755114a07a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:00:59.937913 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd58a10-f3a4-42c8-ab7a-6755114a07a9 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:00:59.938191 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd58a10-f3a4-42c8-ab7a-6755114a07a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:00:59.938459 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd58a10-f3a4-42c8-ab7a-6755114a07a9 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:00:59.949656 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd58a10-f3a4-42c8-ab7a-6755114a07a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:00:59.954438 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-2bd58a10-f3a4-42c8-ab7a-6755114a07a9 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 12:00:59.954438 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:00:59.954438 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:00:59.954438 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:00:59.954438 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:00:59.954438 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:00:59.954438 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:00:59.954438 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:00:59.954438 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:00:59.954438 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:00:59.954438 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:00:59.954438 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:00:59.954438 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:00:59.954438 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:00:59.954438 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:00:59.954438 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:00:59.955629 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:00:59.955629 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:00:59.955629 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:00:59.955629 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:00:59.955629 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:00:59.955629 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:00:59.955629 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:00:59.955629 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:00:59.955629 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:00:59.955629 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:00:59.955629 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:00:59.955629 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 12:00:59.955629 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:00:59.955629 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:00:59.955629 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:00:59.955629 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:00:59.956894 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:00:59.956894 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:00:59.956894 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:00:59.956894 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:00:59.956894 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:00:59.956894 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:00:59.956894 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:00:59.956894 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:00:59.956894 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:00:59.956894 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:00:59.956894 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 12:00:59.956894 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:00:59.959264 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 514/1032] 10.222.0.49 () {56 vars in 1120 bytes} [Fri Sep 9 12:00:59 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:00:59.961501 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2275358c-13a5-4f53-9d7f-c0795a6d9389 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:00.015973 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:00.061660 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2275358c-13a5-4f53-9d7f-c0795a6d9389 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:00.067274 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd58a10-f3a4-42c8-ab7a-6755114a07a9 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:00.098874 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbf215d-7af3-49a5-a140-b8d7c5b324ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:00.099505 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbf215d-7af3-49a5-a140-b8d7c5b324ea None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:00.099784 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbf215d-7af3-49a5-a140-b8d7c5b324ea None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:00.100057 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbf215d-7af3-49a5-a140-b8d7c5b324ea None admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:00.100718 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbf215d-7af3-49a5-a140-b8d7c5b324ea None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:01:00.101017 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbf215d-7af3-49a5-a140-b8d7c5b324ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:00.102378 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbf215d-7af3-49a5-a140-b8d7c5b324ea None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:00.111730 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 519/1033] 10.222.0.49 () {56 vars in 1115 bytes} [Fri Sep 9 12:00:59 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:01:00.117179 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbf215d-7af3-49a5-a140-b8d7c5b324ea None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:00.170146 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd58a10-f3a4-42c8-ab7a-6755114a07a9 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:00.205151 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393df1d9-2443-47eb-acfd-300f50fd24da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:00.205785 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393df1d9-2443-47eb-acfd-300f50fd24da None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:00.206060 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393df1d9-2443-47eb-acfd-300f50fd24da None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:00.206334 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393df1d9-2443-47eb-acfd-300f50fd24da None admin] PATH_INFO: `/v3/domains/default` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:00.207523 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393df1d9-2443-47eb-acfd-300f50fd24da None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:00.208896 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393df1d9-2443-47eb-acfd-300f50fd24da None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:00.215267 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 515/1034] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 12:01:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:01:00.216008 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393df1d9-2443-47eb-acfd-300f50fd24da None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:00.218600 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:00.224474 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbf215d-7af3-49a5-a140-b8d7c5b324ea None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:00.241869 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:00.264884 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559890ca-b547-46a5-be00-0ef4e5f4c4be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:00.265523 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559890ca-b547-46a5-be00-0ef4e5f4c4be None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:00.265804 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559890ca-b547-46a5-be00-0ef4e5f4c4be None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:00.266073 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559890ca-b547-46a5-be00-0ef4e5f4c4be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:00.277881 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559890ca-b547-46a5-be00-0ef4e5f4c4be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:00.279379 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559890ca-b547-46a5-be00-0ef4e5f4c4be None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:00.295588 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-559890ca-b547-46a5-be00-0ef4e5f4c4be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:01:00.295588 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:01:00.295588 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:01:00.295588 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:01:00.295588 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:01:00.295588 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:01:00.295588 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:01:00.295588 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:01:00.295588 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:01:00.295588 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:01:00.295588 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:01:00.295588 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:01:00.295588 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:01:00.295588 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:01:00.295588 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:01:00.295588 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:01:00.296563 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:01:00.296563 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:01:00.296563 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:01:00.296563 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:01:00.296563 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:01:00.296563 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:01:00.296563 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:01:00.296563 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 12:01:00.296563 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:01:00.296563 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:01:00.296563 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:01:00.296563 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:01:00.296563 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:01:00.296563 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:01:00.296563 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:01:00.296563 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:01:00.296563 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:01:00.297451 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:01:00.297451 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:01:00.297451 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:01:00.299444 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 520/1035] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 12:01:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 12:01:00.303225 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559890ca-b547-46a5-be00-0ef4e5f4c4be None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:00.320627 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393df1d9-2443-47eb-acfd-300f50fd24da None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:00.354450 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50a6c79-f44e-42d5-813a-b97810fd9aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:00.355091 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50a6c79-f44e-42d5-813a-b97810fd9aa7 None admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:00.355370 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50a6c79-f44e-42d5-813a-b97810fd9aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:00.355639 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50a6c79-f44e-42d5-813a-b97810fd9aa7 None admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:00.356282 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50a6c79-f44e-42d5-813a-b97810fd9aa7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:00.357712 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50a6c79-f44e-42d5-813a-b97810fd9aa7 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:00.380675 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 516/1036] 10.222.0.49 () {60 vars in 1132 bytes} [Fri Sep 9 12:01:00 2022] POST /identity/v3/users => generated 285 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:01:00.415881 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559890ca-b547-46a5-be00-0ef4e5f4c4be None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:00.455997 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac324cec-9a5a-4c65-842d-b19e41b0a115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:00.456649 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac324cec-9a5a-4c65-842d-b19e41b0a115 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:00.456903 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac324cec-9a5a-4c65-842d-b19e41b0a115 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:00.457176 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac324cec-9a5a-4c65-842d-b19e41b0a115 None admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:00.457921 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac324cec-9a5a-4c65-842d-b19e41b0a115 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:01:00.458171 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac324cec-9a5a-4c65-842d-b19e41b0a115 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:00.459614 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac324cec-9a5a-4c65-842d-b19e41b0a115 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:00.473428 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 521/1037] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 9 12:01:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:01:00.479088 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50a6c79-f44e-42d5-813a-b97810fd9aa7 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:00.536895 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1848cfca-e421-482d-9dbd-d28f8dde4bf5 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:00.537262 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1848cfca-e421-482d-9dbd-d28f8dde4bf5 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:00.537446 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1848cfca-e421-482d-9dbd-d28f8dde4bf5 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:00.539011 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 522/1038] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:01:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 12:01:00.544293 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5577ed74-7983-4a69-95fd-136bac79c13c None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:00.544563 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5577ed74-7983-4a69-95fd-136bac79c13c None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:00.544831 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5577ed74-7983-4a69-95fd-136bac79c13c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:00.546431 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:00.566890 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-5577ed74-7983-4a69-95fd-136bac79c13c None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:01:00.568169 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-5577ed74-7983-4a69-95fd-136bac79c13c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:01:00.598951 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50a6c79-f44e-42d5-813a-b97810fd9aa7 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:00.600783 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:00.647105 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c5c2ad-8680-4854-8673-b0993606a24b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:00.647752 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c5c2ad-8680-4854-8673-b0993606a24b None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:00.648022 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c5c2ad-8680-4854-8673-b0993606a24b None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:00.648469 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c5c2ad-8680-4854-8673-b0993606a24b None admin] PATH_INFO: `/v3/users/nova` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:00.652952 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 523/1039] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:01:00 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 12:01:00.656710 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c5c2ad-8680-4854-8673-b0993606a24b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:00.658304 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c5c2ad-8680-4854-8673-b0993606a24b None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:00.669569 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-f5c5c2ad-8680-4854-8673-b0993606a24b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 12:01:00.669569 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:01:00.669569 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:01:00.669569 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:01:00.669569 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:01:00.669569 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:01:00.669569 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:01:00.669569 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:01:00.669569 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:01:00.669569 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:01:00.669569 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:01:00.669569 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:01:00.669569 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 12:01:00.669569 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 12:01:00.669569 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 12:01:00.669569 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 12:01:00.670590 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:01:00.670590 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:01:00.670590 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 12:01:00.670590 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:01:00.670590 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 12:01:00.670590 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:01:00.670590 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:01:00.670590 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:01:00.670590 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:01:00.670590 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:01:00.670590 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:01:00.670590 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 12:01:00.670590 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:01:00.670590 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:01:00.670590 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:01:00.670590 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:01:00.670590 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:01:00.670590 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:01:00.671696 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:01:00.671696 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:01:00.671696 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 12:01:00.671696 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 12:01:00.671696 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 12:01:00.671696 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 12:01:00.671696 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 12:01:00.671696 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 12:01:00.671696 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 12:01:00.671696 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:01:00.671696 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 517/1040] 10.222.0.49 () {56 vars in 1093 bytes} [Fri Sep 9 12:01:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:01:00.678758 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5577ed74-7983-4a69-95fd-136bac79c13c None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:00.780130 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5577ed74-7983-4a69-95fd-136bac79c13c None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:00.816467 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf68d4c-06e5-441a-a35e-f8301e1f3449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:00.817080 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf68d4c-06e5-441a-a35e-f8301e1f3449 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:00.817353 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf68d4c-06e5-441a-a35e-f8301e1f3449 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:00.817626 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf68d4c-06e5-441a-a35e-f8301e1f3449 None admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:00.818351 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf68d4c-06e5-441a-a35e-f8301e1f3449 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:01:00.818653 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf68d4c-06e5-441a-a35e-f8301e1f3449 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:00.820115 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf68d4c-06e5-441a-a35e-f8301e1f3449 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:00.841963 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 524/1041] 10.222.0.49 () {56 vars in 1133 bytes} [Fri Sep 9 12:01:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:01:00.843252 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c5c2ad-8680-4854-8673-b0993606a24b None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:00.923145 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf68d4c-06e5-441a-a35e-f8301e1f3449 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:00.942628 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:00.955312 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c5c2ad-8680-4854-8673-b0993606a24b None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:00.974177 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:00.992864 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da53070-4fe4-4dab-bb2d-b9f51ced0676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:00.993663 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da53070-4fe4-4dab-bb2d-b9f51ced0676 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:00.993847 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da53070-4fe4-4dab-bb2d-b9f51ced0676 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:00.994078 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da53070-4fe4-4dab-bb2d-b9f51ced0676 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:01.040701 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf68d4c-06e5-441a-a35e-f8301e1f3449 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:01.068481 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da53070-4fe4-4dab-bb2d-b9f51ced0676 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:01.070160 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da53070-4fe4-4dab-bb2d-b9f51ced0676 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:01.075361 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5216eb-813f-4272-83d4-67dacc99965f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:01.075992 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5216eb-813f-4272-83d4-67dacc99965f None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:01.076322 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5216eb-813f-4272-83d4-67dacc99965f None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:01.076630 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5216eb-813f-4272-83d4-67dacc99965f None admin] PATH_INFO: `/v3/services/image` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:01.077250 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5216eb-813f-4272-83d4-67dacc99965f None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:01.078775 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5216eb-813f-4272-83d4-67dacc99965f None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:01.087970 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-ad5216eb-813f-4272-83d4-67dacc99965f None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 12:01:01.087970 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:01:01.087970 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 12:01:01.087970 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 12:01:01.087970 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 12:01:01.087970 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 12:01:01.087970 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 12:01:01.087970 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 12:01:01.087970 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 12:01:01.087970 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:01:01.087970 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 12:01:01.087970 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:01:01.087970 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:01:01.087970 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:01:01.087970 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:01:01.087970 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:01:01.087970 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:01:01.089174 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:01:01.089174 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:01:01.089174 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:01:01.089174 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:01:01.089174 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:01:01.089174 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:01:01.089174 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 12:01:01.089174 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 12:01:01.089174 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 12:01:01.089174 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 12:01:01.089174 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:01:01.089174 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:01:01.089174 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:01:01.089174 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:01:01.089174 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:01:01.089174 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:01:01.090227 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:01:01.090227 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 12:01:01.090227 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:01:01.090227 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:01:01.090227 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:01:01.090227 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:01:01.090227 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:01:01.090227 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:01:01.090227 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:01:01.090227 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:01:01.090227 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 12:01:01.090227 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 12:01:01.090227 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 12:01:01.090227 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:01:01.094020 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 525/1042] 10.222.0.49 () {56 vars in 1105 bytes} [Fri Sep 9 12:01:00 2022] GET /identity/v3/services/image => generated 86 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 12:01:01.094845 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5216eb-813f-4272-83d4-67dacc99965f None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:01.180614 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 518/1043] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 9 12:01:00 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 12:01:01.182722 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:01.186219 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da53070-4fe4-4dab-bb2d-b9f51ced0676 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:01.204229 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:01.204410 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5216eb-813f-4272-83d4-67dacc99965f None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:01.240160 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a232ae8a-e621-4d8c-a0b6-b2e0f625bcdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:01.240776 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a232ae8a-e621-4d8c-a0b6-b2e0f625bcdf None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:01.241058 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a232ae8a-e621-4d8c-a0b6-b2e0f625bcdf None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:01.241330 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a232ae8a-e621-4d8c-a0b6-b2e0f625bcdf None admin] PATH_INFO: `/v3/services` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:01.241961 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a232ae8a-e621-4d8c-a0b6-b2e0f625bcdf None admin] RBAC: Adding query filter params (name=image) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:01:01.242263 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a232ae8a-e621-4d8c-a0b6-b2e0f625bcdf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:01.243659 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a232ae8a-e621-4d8c-a0b6-b2e0f625bcdf None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:01.248748 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 526/1044] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 12:01:01 2022] GET /identity/v3/services?name=image => generated 122 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:01:01.254490 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a232ae8a-e621-4d8c-a0b6-b2e0f625bcdf None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:01.295032 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da53070-4fe4-4dab-bb2d-b9f51ced0676 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:01.335926 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213f8246-2acd-41a4-abf8-9e82267f48fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:01.336911 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213f8246-2acd-41a4-abf8-9e82267f48fd None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:01.337339 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213f8246-2acd-41a4-abf8-9e82267f48fd None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:01.337763 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213f8246-2acd-41a4-abf8-9e82267f48fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:01.349221 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213f8246-2acd-41a4-abf8-9e82267f48fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:01.350978 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213f8246-2acd-41a4-abf8-9e82267f48fd None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:01.360610 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a232ae8a-e621-4d8c-a0b6-b2e0f625bcdf None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:01.367183 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-213f8246-2acd-41a4-abf8-9e82267f48fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:01:01.367183 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:01:01.367183 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:01:01.367183 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:01:01.367183 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:01:01.367183 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:01:01.367183 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:01:01.367183 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:01:01.367183 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:01:01.367183 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:01:01.367183 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:01:01.367183 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:01:01.367183 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:01:01.367183 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:01:01.367183 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:01:01.367183 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:01:01.368327 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:01:01.368327 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:01:01.368327 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:01:01.368327 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:01:01.368327 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:01:01.368327 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:01:01.368327 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:01:01.368327 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 12:01:01.368327 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:01:01.368327 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:01:01.368327 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:01:01.368327 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:01:01.368327 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:01:01.368327 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:01:01.368327 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:01:01.368327 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:01:01.368327 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:01:01.369440 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:01:01.369440 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:01:01.369440 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:01:01.371255 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 519/1045] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 12:01:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:01:01.376523 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213f8246-2acd-41a4-abf8-9e82267f48fd None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:01.400501 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab6116d-67fc-46ee-a9ee-6e9df6b7e8f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:01.401113 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab6116d-67fc-46ee-a9ee-6e9df6b7e8f3 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:01.401503 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab6116d-67fc-46ee-a9ee-6e9df6b7e8f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:01.401776 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab6116d-67fc-46ee-a9ee-6e9df6b7e8f3 None admin] PATH_INFO: `/v3/services` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:01.402519 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab6116d-67fc-46ee-a9ee-6e9df6b7e8f3 None admin] RBAC: Adding query filter params (type=image) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:01:01.402822 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab6116d-67fc-46ee-a9ee-6e9df6b7e8f3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:01.404239 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab6116d-67fc-46ee-a9ee-6e9df6b7e8f3 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:01.414550 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 527/1046] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 12:01:01 2022] GET /identity/v3/services?type=image => generated 350 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:01:01.415488 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab6116d-67fc-46ee-a9ee-6e9df6b7e8f3 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:01.426507 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:01.451722 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:01.485986 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213f8246-2acd-41a4-abf8-9e82267f48fd None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:01.519514 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab6116d-67fc-46ee-a9ee-6e9df6b7e8f3 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:01.521334 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3d3474-5d0c-4487-b9e1-a6d79d7e07a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:01.522064 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3d3474-5d0c-4487-b9e1-a6d79d7e07a6 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:01.522344 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3d3474-5d0c-4487-b9e1-a6d79d7e07a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:01.522618 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3d3474-5d0c-4487-b9e1-a6d79d7e07a6 None admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:01.523307 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3d3474-5d0c-4487-b9e1-a6d79d7e07a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:01:01.523612 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3d3474-5d0c-4487-b9e1-a6d79d7e07a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:01.524988 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3d3474-5d0c-4487-b9e1-a6d79d7e07a6 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:01.542649 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 520/1047] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 9 12:01:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:01:01.543739 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3d3474-5d0c-4487-b9e1-a6d79d7e07a6 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:01.557842 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1611864f-ef52-4fc4-9732-1ace01471377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:01.558388 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1611864f-ef52-4fc4-9732-1ace01471377 None admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:01.558662 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1611864f-ef52-4fc4-9732-1ace01471377 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:01.558960 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1611864f-ef52-4fc4-9732-1ace01471377 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:01.559677 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1611864f-ef52-4fc4-9732-1ace01471377 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:01.560984 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1611864f-ef52-4fc4-9732-1ace01471377 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:01.588566 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 528/1048] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:01:01 2022] POST /identity/v3/endpoints => generated 327 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:01:01.672423 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3d3474-5d0c-4487-b9e1-a6d79d7e07a6 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:01.706738 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:01.720544 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d83af7f-caae-4bd9-9ac8-9abec47fc442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:01.721237 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d83af7f-caae-4bd9-9ac8-9abec47fc442 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:01.721521 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d83af7f-caae-4bd9-9ac8-9abec47fc442 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:01.721791 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d83af7f-caae-4bd9-9ac8-9abec47fc442 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:01.729261 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d83af7f-caae-4bd9-9ac8-9abec47fc442 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:01.730735 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d83af7f-caae-4bd9-9ac8-9abec47fc442 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:01.743940 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-5d83af7f-caae-4bd9-9ac8-9abec47fc442 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:01:01.743940 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:01:01.743940 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:01:01.743940 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:01:01.743940 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:01:01.743940 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:01:01.743940 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:01:01.743940 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:01:01.743940 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:01:01.743940 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:01:01.743940 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:01:01.743940 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:01:01.743940 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 12:01:01.743940 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 12:01:01.743940 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 12:01:01.744917 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 12:01:01.744917 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:01:01.744917 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:01:01.744917 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:01:01.744917 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:01:01.744917 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:01:01.744917 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:01:01.744917 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:01:01.744917 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 12:01:01.744917 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:01:01.744917 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:01:01.744917 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:01:01.744917 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:01:01.744917 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:01:01.744917 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:01:01.744917 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:01:01.744917 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:01:01.750733 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 12:01:01.750733 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 12:01:01.750733 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 12:01:01.750733 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 12:01:01.750733 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 12:01:01.750733 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 12:01:01.750733 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:01:01.750733 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:01:01.750733 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 521/1049] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 9 12:01:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:01:01.751909 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1611864f-ef52-4fc4-9732-1ace01471377 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:01.818716 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:01.864113 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1611864f-ef52-4fc4-9732-1ace01471377 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:01.920610 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fb7647-f2f8-4cc1-81ce-c4fe76410304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:01.921636 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fb7647-f2f8-4cc1-81ce-c4fe76410304 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:01.922127 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fb7647-f2f8-4cc1-81ce-c4fe76410304 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:01.922618 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fb7647-f2f8-4cc1-81ce-c4fe76410304 None admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:01.923964 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fb7647-f2f8-4cc1-81ce-c4fe76410304 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:01:01.924510 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fb7647-f2f8-4cc1-81ce-c4fe76410304 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:01.927207 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fb7647-f2f8-4cc1-81ce-c4fe76410304 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:01.949213 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 529/1050] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Sep 9 12:01:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:01:01.956181 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d83af7f-caae-4bd9-9ac8-9abec47fc442 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:02.057648 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d83af7f-caae-4bd9-9ac8-9abec47fc442 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:02.092103 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c13b494-f6d2-4525-b5a5-a1fb57a900a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:02.092631 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c13b494-f6d2-4525-b5a5-a1fb57a900a4 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:02.092908 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c13b494-f6d2-4525-b5a5-a1fb57a900a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:02.093178 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c13b494-f6d2-4525-b5a5-a1fb57a900a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:02.093884 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c13b494-f6d2-4525-b5a5-a1fb57a900a4 None admin] RBAC: Adding query filter params (role.id=db537712f45d4e6980592d3ddefb312a, scope.project.id=fe863fcd75f1406ebbffc5016078f73e, user.id=70bac169d5cd43e1926f310344ac11fa) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:01:02.094187 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c13b494-f6d2-4525-b5a5-a1fb57a900a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:02.096403 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c13b494-f6d2-4525-b5a5-a1fb57a900a4 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:02.102059 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 522/1051] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Sep 9 12:01:01 2022] GET /identity/v3/role_assignments?user.id=70bac169d5cd43e1926f310344ac11fa&scope.project.id=fe863fcd75f1406ebbffc5016078f73e&role.id=db537712f45d4e6980592d3ddefb312a => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:01:02.769808 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6f3f93-12fb-4e0d-846a-a953410a5921 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:02.770353 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6f3f93-12fb-4e0d-846a-a953410a5921 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:02.770353 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6f3f93-12fb-4e0d-846a-a953410a5921 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:02.771776 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 530/1052] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:01:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:01:02.777167 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f1535d-43ce-40a6-a2d7-16d4191b35c7 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:02.777588 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f1535d-43ce-40a6-a2d7-16d4191b35c7 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:02.777753 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f1535d-43ce-40a6-a2d7-16d4191b35c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:02.795692 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-e6f1535d-43ce-40a6-a2d7-16d4191b35c7 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:01:02.796881 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-e6f1535d-43ce-40a6-a2d7-16d4191b35c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:01:02.852815 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:02.875509 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 523/1053] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:01:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 12:01:03.143601 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6f3f93-12fb-4e0d-846a-a953410a5921 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:03.242320 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6f3f93-12fb-4e0d-846a-a953410a5921 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:03.265437 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:03.277652 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8762227-a6af-4bdb-ac97-91b190b22d77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:03.278288 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8762227-a6af-4bdb-ac97-91b190b22d77 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:03.278562 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8762227-a6af-4bdb-ac97-91b190b22d77 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:03.278826 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8762227-a6af-4bdb-ac97-91b190b22d77 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:03.289005 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8762227-a6af-4bdb-ac97-91b190b22d77 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:01:03.293756 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-a8762227-a6af-4bdb-ac97-91b190b22d77 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 12:01:03.293756 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:01:03.293756 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:01:03.293756 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:01:03.293756 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:01:03.293756 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:01:03.293756 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:01:03.293756 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:01:03.293756 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:01:03.293756 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:01:03.293756 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:01:03.293756 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:01:03.293756 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:01:03.293756 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:01:03.293756 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:01:03.293756 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:01:03.294705 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:01:03.294705 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:01:03.294705 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:01:03.294705 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:01:03.294705 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:01:03.294705 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:01:03.294705 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:01:03.294705 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:01:03.294705 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:01:03.294705 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:01:03.294705 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:01:03.294705 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 12:01:03.294705 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:01:03.294705 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:01:03.294705 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:01:03.294705 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:01:03.295871 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:01:03.295871 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:01:03.295871 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:01:03.295871 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:01:03.295871 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:01:03.295871 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:01:03.295871 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:01:03.295871 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:01:03.295871 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:01:03.295871 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:01:03.295871 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 12:01:03.295871 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:01:03.299034 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 531/1054] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Sep 9 12:01:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 12:01:03.301281 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f1535d-43ce-40a6-a2d7-16d4191b35c7 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:03.404697 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f1535d-43ce-40a6-a2d7-16d4191b35c7 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:03.440379 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92278b5-f19c-40af-a8c1-d1525aefc9dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:03.440982 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92278b5-f19c-40af-a8c1-d1525aefc9dd None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:03.441256 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92278b5-f19c-40af-a8c1-d1525aefc9dd None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:03.441521 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92278b5-f19c-40af-a8c1-d1525aefc9dd None admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:03.442172 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92278b5-f19c-40af-a8c1-d1525aefc9dd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:01:03.442472 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92278b5-f19c-40af-a8c1-d1525aefc9dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:03.443925 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92278b5-f19c-40af-a8c1-d1525aefc9dd None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:03.456802 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 524/1055] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 9 12:01:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:01:03.458433 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8762227-a6af-4bdb-ac97-91b190b22d77 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:03.559288 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8762227-a6af-4bdb-ac97-91b190b22d77 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:03.593134 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628f7ecc-050a-4693-8766-eb2d95a9f35c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:03.593760 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628f7ecc-050a-4693-8766-eb2d95a9f35c None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:03.594036 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628f7ecc-050a-4693-8766-eb2d95a9f35c None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:03.594302 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628f7ecc-050a-4693-8766-eb2d95a9f35c None admin] PATH_INFO: `/v3/users/cce103ff40d04e67b820e5d1b045e14d` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:03.608931 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628f7ecc-050a-4693-8766-eb2d95a9f35c None admin] RBAC: Authorizing `identity:get_user(user_id=cce103ff40d04e67b820e5d1b045e14d)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:03.610372 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628f7ecc-050a-4693-8766-eb2d95a9f35c None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:03.612733 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 532/1056] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Sep 9 12:01:03 2022] GET /identity/v3/users/cce103ff40d04e67b820e5d1b045e14d => generated 285 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:01:03.618365 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92278b5-f19c-40af-a8c1-d1525aefc9dd None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:03.666162 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:03.727114 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92278b5-f19c-40af-a8c1-d1525aefc9dd None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:03.762458 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df625dbe-ec3b-4bda-b207-160f4e8f83f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:03.763119 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df625dbe-ec3b-4bda-b207-160f4e8f83f7 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:03.763398 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df625dbe-ec3b-4bda-b207-160f4e8f83f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:03.763668 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df625dbe-ec3b-4bda-b207-160f4e8f83f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:03.836418 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df625dbe-ec3b-4bda-b207-160f4e8f83f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:03.837867 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df625dbe-ec3b-4bda-b207-160f4e8f83f7 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:03.881194 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:03.882304 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a599503e-d0ee-4627-a945-3de199778de0 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:03.882818 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a599503e-d0ee-4627-a945-3de199778de0 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:03.882818 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a599503e-d0ee-4627-a945-3de199778de0 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:03.884504 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 533/1057] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:01:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 12:01:03.890615 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7266e9d9-bd7c-41d9-b9c5-f9151e2ff7b6 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:03.891122 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7266e9d9-bd7c-41d9-b9c5-f9151e2ff7b6 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:03.891544 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7266e9d9-bd7c-41d9-b9c5-f9151e2ff7b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:03.910810 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-7266e9d9-bd7c-41d9-b9c5-f9151e2ff7b6 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:01:03.912300 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-7266e9d9-bd7c-41d9-b9c5-f9151e2ff7b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:01:03.946287 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 525/1058] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 9 12:01:03 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 12:01:03.950974 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:03.952035 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df625dbe-ec3b-4bda-b207-160f4e8f83f7 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:03.987085 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 534/1059] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:01:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 12:01:04.054636 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df625dbe-ec3b-4bda-b207-160f4e8f83f7 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:04.088858 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e46fbd8-af55-4b9f-a163-c435562a5eca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:04.089490 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e46fbd8-af55-4b9f-a163-c435562a5eca None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:04.089766 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e46fbd8-af55-4b9f-a163-c435562a5eca None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:04.090034 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e46fbd8-af55-4b9f-a163-c435562a5eca None admin] PATH_INFO: `/v3/projects/b427ec62a8c545a68a0ce1206bc0787b` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:04.091195 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e46fbd8-af55-4b9f-a163-c435562a5eca None admin] RBAC: Authorizing `identity:get_project(project_id=b427ec62a8c545a68a0ce1206bc0787b)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:04.092623 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e46fbd8-af55-4b9f-a163-c435562a5eca None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:04.094418 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 526/1060] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Sep 9 12:01:03 2022] GET /identity/v3/projects/b427ec62a8c545a68a0ce1206bc0787b => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:01:04.101098 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7266e9d9-bd7c-41d9-b9c5-f9151e2ff7b6 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:04.206662 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7266e9d9-bd7c-41d9-b9c5-f9151e2ff7b6 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:04.240271 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f18bb1b-be89-48af-98e7-5eff3edec57e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:04.240931 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f18bb1b-be89-48af-98e7-5eff3edec57e None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:04.241206 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f18bb1b-be89-48af-98e7-5eff3edec57e None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:04.241472 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f18bb1b-be89-48af-98e7-5eff3edec57e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:04.242233 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f18bb1b-be89-48af-98e7-5eff3edec57e None admin] RBAC: Adding query filter params (role.id=3f2ca33902a04b6dbc2ef7d06d7289d3, scope.project.id=b427ec62a8c545a68a0ce1206bc0787b, user.id=cce103ff40d04e67b820e5d1b045e14d) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:01:04.242540 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f18bb1b-be89-48af-98e7-5eff3edec57e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:04.244000 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f18bb1b-be89-48af-98e7-5eff3edec57e None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:04.255173 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 535/1061] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Sep 9 12:01:04 2022] GET /identity/v3/role_assignments?user.id=cce103ff40d04e67b820e5d1b045e14d&scope.project.id=b427ec62a8c545a68a0ce1206bc0787b&role.id=3f2ca33902a04b6dbc2ef7d06d7289d3 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:01:04.270410 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e46fbd8-af55-4b9f-a163-c435562a5eca None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:04.275996 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0144fe-94a0-4644-a4b2-f6069168f645 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:04.276333 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0144fe-94a0-4644-a4b2-f6069168f645 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:04.276555 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0144fe-94a0-4644-a4b2-f6069168f645 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:04.278121 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 536/1062] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:01:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 12:01:04.283923 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482ab7ec-a8f6-4d30-b2cb-ccb62c11a23d None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:04.284271 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:04.284380 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482ab7ec-a8f6-4d30-b2cb-ccb62c11a23d None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:04.285015 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482ab7ec-a8f6-4d30-b2cb-ccb62c11a23d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:04.305048 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-482ab7ec-a8f6-4d30-b2cb-ccb62c11a23d None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:01:04.306299 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-482ab7ec-a8f6-4d30-b2cb-ccb62c11a23d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:01:04.341671 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:04.383494 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e46fbd8-af55-4b9f-a163-c435562a5eca None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:04.390503 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 537/1063] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:01:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 12:01:04.425293 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62349302-f408-4f90-b26c-6e490992c130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:04.425901 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62349302-f408-4f90-b26c-6e490992c130 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:04.426225 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62349302-f408-4f90-b26c-6e490992c130 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:04.426491 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62349302-f408-4f90-b26c-6e490992c130 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:04.497582 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62349302-f408-4f90-b26c-6e490992c130 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:04.498947 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62349302-f408-4f90-b26c-6e490992c130 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:04.510322 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:04.600348 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 527/1064] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 9 12:01:04 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 12:01:04.608225 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482ab7ec-a8f6-4d30-b2cb-ccb62c11a23d None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:04.672082 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62349302-f408-4f90-b26c-6e490992c130 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:04.716797 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482ab7ec-a8f6-4d30-b2cb-ccb62c11a23d None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:04.755320 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feb7eee-6af5-4b1d-b98c-4c42f5787040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:04.755956 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feb7eee-6af5-4b1d-b98c-4c42f5787040 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:04.756229 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feb7eee-6af5-4b1d-b98c-4c42f5787040 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:04.756505 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feb7eee-6af5-4b1d-b98c-4c42f5787040 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:04.767640 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1feb7eee-6af5-4b1d-b98c-4c42f5787040 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:04.769089 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1feb7eee-6af5-4b1d-b98c-4c42f5787040 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:04.782299 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62349302-f408-4f90-b26c-6e490992c130 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:04.784593 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-1feb7eee-6af5-4b1d-b98c-4c42f5787040 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:01:04.784593 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:01:04.784593 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:01:04.784593 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:01:04.784593 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:01:04.784593 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:01:04.784593 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:01:04.784593 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:01:04.784593 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:01:04.784593 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:01:04.784593 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:01:04.784593 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:01:04.784593 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:01:04.784593 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:01:04.784593 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:01:04.784593 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:01:04.785610 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:01:04.785610 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:01:04.785610 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:01:04.785610 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:01:04.785610 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:01:04.785610 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:01:04.785610 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:01:04.785610 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 12:01:04.785610 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:01:04.785610 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:01:04.785610 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:01:04.785610 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:01:04.785610 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:01:04.785610 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:01:04.785610 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:01:04.785610 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:01:04.785610 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:01:04.786503 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:01:04.786503 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:01:04.786503 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:01:04.790669 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 538/1065] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 12:01:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:01:04.791513 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feb7eee-6af5-4b1d-b98c-4c42f5787040 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:04.797327 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:04.812142 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:04.825363 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f92bce-bc35-416e-baa9-9b6c87001a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:04.825988 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f92bce-bc35-416e-baa9-9b6c87001a78 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:04.826268 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f92bce-bc35-416e-baa9-9b6c87001a78 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:04.826531 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f92bce-bc35-416e-baa9-9b6c87001a78 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:04.836892 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f92bce-bc35-416e-baa9-9b6c87001a78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:01:04.841614 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-d1f92bce-bc35-416e-baa9-9b6c87001a78 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 12:01:04.841614 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:01:04.841614 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:01:04.841614 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:01:04.841614 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:01:04.841614 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:01:04.841614 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:01:04.841614 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:01:04.841614 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:01:04.841614 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:01:04.841614 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:01:04.841614 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:01:04.841614 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:01:04.841614 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:01:04.841614 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:01:04.841614 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:01:04.842549 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:01:04.842549 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:01:04.842549 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:01:04.842549 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:01:04.842549 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:01:04.842549 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:01:04.842549 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:01:04.842549 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:01:04.842549 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:01:04.842549 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:01:04.842549 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:01:04.842549 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 12:01:04.842549 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:01:04.842549 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:01:04.842549 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:01:04.842549 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:01:04.843425 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:01:04.843425 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:01:04.843425 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:01:04.843425 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:01:04.843425 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:01:04.843425 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:01:04.843425 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:01:04.843425 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:01:04.843425 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:01:04.843425 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:01:04.843425 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 12:01:04.843425 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:01:04.843425 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 528/1066] 10.222.0.49 () {56 vars in 1120 bytes} [Fri Sep 9 12:01:04 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:01:04.849299 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f92bce-bc35-416e-baa9-9b6c87001a78 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:04.901300 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feb7eee-6af5-4b1d-b98c-4c42f5787040 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:04.936549 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5187981-5764-431e-947f-40e056a2fe55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:04.937165 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5187981-5764-431e-947f-40e056a2fe55 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:04.937439 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5187981-5764-431e-947f-40e056a2fe55 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:04.937705 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5187981-5764-431e-947f-40e056a2fe55 None admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:04.938336 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5187981-5764-431e-947f-40e056a2fe55 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:01:04.938757 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5187981-5764-431e-947f-40e056a2fe55 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:04.940874 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5187981-5764-431e-947f-40e056a2fe55 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:04.953188 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 539/1067] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 9 12:01:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:01:04.966223 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f92bce-bc35-416e-baa9-9b6c87001a78 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:05.006554 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6a82a1-a156-49b7-9e6a-22cd24b6b7b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:05.007141 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6a82a1-a156-49b7-9e6a-22cd24b6b7b5 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:05.007418 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6a82a1-a156-49b7-9e6a-22cd24b6b7b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:05.007686 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6a82a1-a156-49b7-9e6a-22cd24b6b7b5 None admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:05.008449 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6a82a1-a156-49b7-9e6a-22cd24b6b7b5 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:01:05.008754 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6a82a1-a156-49b7-9e6a-22cd24b6b7b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:05.010215 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6a82a1-a156-49b7-9e6a-22cd24b6b7b5 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:05.025024 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 529/1068] 10.222.0.49 () {56 vars in 1115 bytes} [Fri Sep 9 12:01:04 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:01:05.026789 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5187981-5764-431e-947f-40e056a2fe55 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:05.116289 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:05.143025 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5187981-5764-431e-947f-40e056a2fe55 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:05.180912 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e661e90a-9e91-4b74-bfc9-afc3e65f2c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:05.181539 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e661e90a-9e91-4b74-bfc9-afc3e65f2c70 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:05.181811 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e661e90a-9e91-4b74-bfc9-afc3e65f2c70 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:05.182076 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e661e90a-9e91-4b74-bfc9-afc3e65f2c70 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:05.193234 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e661e90a-9e91-4b74-bfc9-afc3e65f2c70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:05.194702 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e661e90a-9e91-4b74-bfc9-afc3e65f2c70 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:05.209675 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-e661e90a-9e91-4b74-bfc9-afc3e65f2c70 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:01:05.209675 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:01:05.209675 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:01:05.209675 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:01:05.209675 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:01:05.209675 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:01:05.209675 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:01:05.209675 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:01:05.209675 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:01:05.209675 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:01:05.209675 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:01:05.209675 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:01:05.209675 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:01:05.209675 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:01:05.209675 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:01:05.209675 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:01:05.210676 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:01:05.210676 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:01:05.210676 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:01:05.210676 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:01:05.210676 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:01:05.210676 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:01:05.210676 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:01:05.210676 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 12:01:05.210676 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:01:05.210676 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:01:05.210676 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:01:05.210676 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:01:05.210676 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:01:05.210676 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:01:05.210676 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:01:05.210676 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:01:05.210676 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:01:05.211624 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:01:05.211624 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:01:05.211624 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:01:05.211624 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 540/1069] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 12:01:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 12:01:05.216583 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6a82a1-a156-49b7-9e6a-22cd24b6b7b5 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:05.252085 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:05.324672 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6a82a1-a156-49b7-9e6a-22cd24b6b7b5 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:05.357550 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa318ac8-e853-4c09-a29c-de5b97444d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:05.358150 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa318ac8-e853-4c09-a29c-de5b97444d7e None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:05.358426 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa318ac8-e853-4c09-a29c-de5b97444d7e None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:05.358697 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa318ac8-e853-4c09-a29c-de5b97444d7e None admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:05.359366 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa318ac8-e853-4c09-a29c-de5b97444d7e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:01:05.359668 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa318ac8-e853-4c09-a29c-de5b97444d7e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:05.361335 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa318ac8-e853-4c09-a29c-de5b97444d7e None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:05.372806 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 530/1070] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 9 12:01:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:01:05.379049 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e661e90a-9e91-4b74-bfc9-afc3e65f2c70 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:05.476844 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e661e90a-9e91-4b74-bfc9-afc3e65f2c70 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:05.504752 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:05.510919 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26265dc-86f0-4a23-90bc-29a69f0d5b07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:05.511546 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26265dc-86f0-4a23-90bc-29a69f0d5b07 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:05.511824 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26265dc-86f0-4a23-90bc-29a69f0d5b07 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:05.512095 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26265dc-86f0-4a23-90bc-29a69f0d5b07 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:05.520701 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26265dc-86f0-4a23-90bc-29a69f0d5b07 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:05.522131 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26265dc-86f0-4a23-90bc-29a69f0d5b07 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:05.533623 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-d26265dc-86f0-4a23-90bc-29a69f0d5b07 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 12:01:05.533623 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:01:05.533623 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:01:05.533623 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:01:05.533623 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:01:05.533623 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:01:05.533623 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:01:05.533623 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:01:05.533623 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:01:05.533623 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:01:05.533623 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:01:05.533623 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:01:05.533623 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 12:01:05.533623 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 12:01:05.533623 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 12:01:05.533623 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 12:01:05.534602 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:01:05.534602 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:01:05.534602 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 12:01:05.534602 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:01:05.534602 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 12:01:05.534602 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:01:05.534602 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:01:05.534602 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:01:05.534602 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:01:05.534602 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:01:05.534602 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:01:05.534602 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 12:01:05.534602 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:01:05.534602 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:01:05.534602 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:01:05.534602 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:01:05.534602 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:01:05.534602 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:01:05.535582 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:01:05.535582 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:01:05.535582 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 12:01:05.535582 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 12:01:05.535582 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 12:01:05.535582 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 12:01:05.535582 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 12:01:05.535582 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 12:01:05.535582 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 12:01:05.535582 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:01:05.535582 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 541/1071] 10.222.0.49 () {56 vars in 1093 bytes} [Fri Sep 9 12:01:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:01:05.540529 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa318ac8-e853-4c09-a29c-de5b97444d7e None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:05.636032 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:05.647391 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa318ac8-e853-4c09-a29c-de5b97444d7e None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:05.681132 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226eaf30-9488-4a83-b06c-8f3abf65f990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:05.681726 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226eaf30-9488-4a83-b06c-8f3abf65f990 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:05.681999 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226eaf30-9488-4a83-b06c-8f3abf65f990 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:05.682279 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226eaf30-9488-4a83-b06c-8f3abf65f990 None admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:05.683009 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226eaf30-9488-4a83-b06c-8f3abf65f990 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:01:05.683308 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226eaf30-9488-4a83-b06c-8f3abf65f990 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:05.684713 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226eaf30-9488-4a83-b06c-8f3abf65f990 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:05.702148 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 531/1072] 10.222.0.49 () {56 vars in 1133 bytes} [Fri Sep 9 12:01:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:01:05.707977 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26265dc-86f0-4a23-90bc-29a69f0d5b07 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:05.804912 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26265dc-86f0-4a23-90bc-29a69f0d5b07 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:05.838337 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5258f4ea-0983-447a-abb1-5ca22880006a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:05.838983 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5258f4ea-0983-447a-abb1-5ca22880006a None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:05.839257 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5258f4ea-0983-447a-abb1-5ca22880006a None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:05.839518 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5258f4ea-0983-447a-abb1-5ca22880006a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:05.849959 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5258f4ea-0983-447a-abb1-5ca22880006a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:05.851416 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5258f4ea-0983-447a-abb1-5ca22880006a None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:05.865749 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-5258f4ea-0983-447a-abb1-5ca22880006a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:01:05.865749 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:01:05.865749 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:01:05.865749 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:01:05.865749 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:01:05.865749 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:01:05.865749 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:01:05.865749 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:01:05.865749 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:01:05.865749 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:01:05.865749 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:01:05.865749 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:01:05.865749 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:01:05.865749 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:01:05.865749 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:01:05.865749 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:01:05.866964 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:01:05.866964 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:01:05.866964 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:01:05.866964 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:01:05.866964 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:01:05.866964 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:01:05.866964 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:01:05.866964 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 12:01:05.866964 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:01:05.866964 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:01:05.866964 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:01:05.866964 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:01:05.866964 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:01:05.866964 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:01:05.866964 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:01:05.866964 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:01:05.866964 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:01:05.867878 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:01:05.867878 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:01:05.867878 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:01:05.867878 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 542/1073] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 12:01:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 12:01:05.872687 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226eaf30-9488-4a83-b06c-8f3abf65f990 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:05.971908 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226eaf30-9488-4a83-b06c-8f3abf65f990 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:06.004453 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b98c6ef-2d9f-4c6a-bd6b-ac7e67fb8a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:06.005049 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b98c6ef-2d9f-4c6a-bd6b-ac7e67fb8a42 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:06.005321 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b98c6ef-2d9f-4c6a-bd6b-ac7e67fb8a42 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:06.005593 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b98c6ef-2d9f-4c6a-bd6b-ac7e67fb8a42 None admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:06.006225 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b98c6ef-2d9f-4c6a-bd6b-ac7e67fb8a42 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:01:06.006525 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b98c6ef-2d9f-4c6a-bd6b-ac7e67fb8a42 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:06.007913 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b98c6ef-2d9f-4c6a-bd6b-ac7e67fb8a42 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:06.019469 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 532/1074] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 9 12:01:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:01:06.025116 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5258f4ea-0983-447a-abb1-5ca22880006a None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:06.068292 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:06.124105 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5258f4ea-0983-447a-abb1-5ca22880006a None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:06.157046 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4a7150-969e-49c9-b210-30534a73cfc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:06.157705 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4a7150-969e-49c9-b210-30534a73cfc7 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:06.158157 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4a7150-969e-49c9-b210-30534a73cfc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:06.158429 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4a7150-969e-49c9-b210-30534a73cfc7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:06.164850 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4a7150-969e-49c9-b210-30534a73cfc7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:06.166277 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4a7150-969e-49c9-b210-30534a73cfc7 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:06.177319 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-6a4a7150-969e-49c9-b210-30534a73cfc7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:01:06.177319 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:01:06.177319 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:01:06.177319 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:01:06.177319 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:01:06.177319 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:01:06.177319 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:01:06.177319 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:01:06.177319 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:01:06.177319 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:01:06.177319 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:01:06.177319 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:01:06.177319 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 12:01:06.177319 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 12:01:06.177319 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 12:01:06.178237 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 12:01:06.178237 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:01:06.178237 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:01:06.178237 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:01:06.178237 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:01:06.178237 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:01:06.178237 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:01:06.178237 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:01:06.178237 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 12:01:06.178237 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:01:06.178237 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:01:06.178237 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:01:06.178237 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:01:06.178237 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:01:06.178237 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:01:06.178237 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:01:06.178237 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:01:06.179632 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 12:01:06.179632 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 12:01:06.179632 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 12:01:06.179632 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 12:01:06.179632 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 12:01:06.179632 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 12:01:06.179632 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:01:06.179632 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:01:06.179632 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 543/1075] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 9 12:01:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:01:06.184236 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b98c6ef-2d9f-4c6a-bd6b-ac7e67fb8a42 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:06.197830 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:06.285873 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b98c6ef-2d9f-4c6a-bd6b-ac7e67fb8a42 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:06.322618 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae20406-d0ca-4e3d-bed8-f9cee5f91bb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:06.323262 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae20406-d0ca-4e3d-bed8-f9cee5f91bb4 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:06.323541 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae20406-d0ca-4e3d-bed8-f9cee5f91bb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:06.323807 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae20406-d0ca-4e3d-bed8-f9cee5f91bb4 None admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:06.324503 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae20406-d0ca-4e3d-bed8-f9cee5f91bb4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:01:06.324804 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae20406-d0ca-4e3d-bed8-f9cee5f91bb4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:06.326225 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae20406-d0ca-4e3d-bed8-f9cee5f91bb4 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:06.338695 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 533/1076] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Sep 9 12:01:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:01:06.344651 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4a7150-969e-49c9-b210-30534a73cfc7 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:06.409890 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0b17fd-6ba0-4f55-8c92-c0148fc16755 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:06.410274 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0b17fd-6ba0-4f55-8c92-c0148fc16755 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:06.410546 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0b17fd-6ba0-4f55-8c92-c0148fc16755 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:06.411900 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 534/1077] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:01:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 12:01:06.416794 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62c93e9-1deb-4731-9270-4e2ef4d47a30 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:06.417070 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62c93e9-1deb-4731-9270-4e2ef4d47a30 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:06.417333 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62c93e9-1deb-4731-9270-4e2ef4d47a30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:06.435577 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-b62c93e9-1deb-4731-9270-4e2ef4d47a30 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:01:06.436770 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-b62c93e9-1deb-4731-9270-4e2ef4d47a30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:01:06.443710 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4a7150-969e-49c9-b210-30534a73cfc7 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:06.445223 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:06.478257 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c8ac9c-eb60-4fb4-ae3f-4d40f0642e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:06.478983 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c8ac9c-eb60-4fb4-ae3f-4d40f0642e81 None admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:06.479207 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c8ac9c-eb60-4fb4-ae3f-4d40f0642e81 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:06.479475 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c8ac9c-eb60-4fb4-ae3f-4d40f0642e81 None admin] PATH_INFO: `/v3/projects/fe863fcd75f1406ebbffc5016078f73e/users/70bac169d5cd43e1926f310344ac11fa/roles/db537712f45d4e6980592d3ddefb312a` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:06.481577 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c8ac9c-eb60-4fb4-ae3f-4d40f0642e81 None admin] RBAC: Authorizing `identity:create_grant(project_id=fe863fcd75f1406ebbffc5016078f73e, user_id=70bac169d5cd43e1926f310344ac11fa, role_id=db537712f45d4e6980592d3ddefb312a)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:06.483030 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c8ac9c-eb60-4fb4-ae3f-4d40f0642e81 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:06.496471 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 544/1078] 10.222.0.49 () {58 vars in 1439 bytes} [Fri Sep 9 12:01:06 2022] PUT /identity/v3/projects/fe863fcd75f1406ebbffc5016078f73e/users/70bac169d5cd43e1926f310344ac11fa/roles/db537712f45d4e6980592d3ddefb312a => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:01:06.503617 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:06.509381 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 535/1079] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:01:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 12:01:06.778172 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c8ac9c-eb60-4fb4-ae3f-4d40f0642e81 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:06.876811 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c8ac9c-eb60-4fb4-ae3f-4d40f0642e81 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:06.909125 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f884cb-51b5-43db-a1e8-8cc5d96a2b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:06.909684 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f884cb-51b5-43db-a1e8-8cc5d96a2b95 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:06.909961 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f884cb-51b5-43db-a1e8-8cc5d96a2b95 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:06.910230 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f884cb-51b5-43db-a1e8-8cc5d96a2b95 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:06.920916 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f884cb-51b5-43db-a1e8-8cc5d96a2b95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:01:06.926411 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-f3f884cb-51b5-43db-a1e8-8cc5d96a2b95 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 12:01:06.926411 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:01:06.926411 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:01:06.926411 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:01:06.926411 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:01:06.926411 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:01:06.926411 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:01:06.926411 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:01:06.926411 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:01:06.926411 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:01:06.926411 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:01:06.926411 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:01:06.926411 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:01:06.926411 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:01:06.926411 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:01:06.926411 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:01:06.927653 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:01:06.927653 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:01:06.927653 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:01:06.927653 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:01:06.927653 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:01:06.927653 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:01:06.927653 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:01:06.927653 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:01:06.927653 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:01:06.927653 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:01:06.927653 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:01:06.927653 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 12:01:06.927653 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:01:06.927653 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:01:06.927653 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:01:06.927653 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:01:06.928565 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:01:06.928565 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:01:06.928565 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:01:06.928565 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:01:06.928565 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:01:06.928565 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:01:06.928565 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:01:06.928565 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:01:06.928565 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:01:06.928565 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:01:06.928565 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 12:01:06.928565 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:01:06.928565 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 545/1080] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Sep 9 12:01:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:01:06.933630 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62c93e9-1deb-4731-9270-4e2ef4d47a30 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:07.031424 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62c93e9-1deb-4731-9270-4e2ef4d47a30 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:07.065421 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df40c9f-8d40-42ff-87c6-4f0cf64b33f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:07.066017 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df40c9f-8d40-42ff-87c6-4f0cf64b33f8 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:07.066292 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df40c9f-8d40-42ff-87c6-4f0cf64b33f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:07.066558 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df40c9f-8d40-42ff-87c6-4f0cf64b33f8 None admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:07.067240 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df40c9f-8d40-42ff-87c6-4f0cf64b33f8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:01:07.067539 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df40c9f-8d40-42ff-87c6-4f0cf64b33f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:07.068898 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df40c9f-8d40-42ff-87c6-4f0cf64b33f8 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:07.076811 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 536/1081] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 9 12:01:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:01:07.083497 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f884cb-51b5-43db-a1e8-8cc5d96a2b95 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:07.109097 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:07.188440 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f884cb-51b5-43db-a1e8-8cc5d96a2b95 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:07.224480 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1e962e-b7be-459c-bff5-96c233c8f1be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:07.225104 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1e962e-b7be-459c-bff5-96c233c8f1be None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:07.225376 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1e962e-b7be-459c-bff5-96c233c8f1be None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:07.225641 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1e962e-b7be-459c-bff5-96c233c8f1be None admin] PATH_INFO: `/v3/users/cce103ff40d04e67b820e5d1b045e14d` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:07.226743 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1e962e-b7be-459c-bff5-96c233c8f1be None admin] RBAC: Authorizing `identity:get_user(user_id=cce103ff40d04e67b820e5d1b045e14d)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:07.228193 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1e962e-b7be-459c-bff5-96c233c8f1be None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:07.229767 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 546/1082] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Sep 9 12:01:07 2022] GET /identity/v3/users/cce103ff40d04e67b820e5d1b045e14d => generated 285 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:01:07.235690 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df40c9f-8d40-42ff-87c6-4f0cf64b33f8 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:07.295948 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:07.336016 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df40c9f-8d40-42ff-87c6-4f0cf64b33f8 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:07.370964 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5f9e60-8c6f-4790-9794-6d8223585416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:07.371603 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5f9e60-8c6f-4790-9794-6d8223585416 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:07.371877 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5f9e60-8c6f-4790-9794-6d8223585416 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:07.372144 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5f9e60-8c6f-4790-9794-6d8223585416 None admin] PATH_INFO: `/v3/projects/b427ec62a8c545a68a0ce1206bc0787b` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:07.373266 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5f9e60-8c6f-4790-9794-6d8223585416 None admin] RBAC: Authorizing `identity:get_project(project_id=b427ec62a8c545a68a0ce1206bc0787b)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:07.374686 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5f9e60-8c6f-4790-9794-6d8223585416 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:07.377280 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 537/1083] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Sep 9 12:01:07 2022] GET /identity/v3/projects/b427ec62a8c545a68a0ce1206bc0787b => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:01:07.382958 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1e962e-b7be-459c-bff5-96c233c8f1be None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:07.472381 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:07.482498 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1e962e-b7be-459c-bff5-96c233c8f1be None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:07.515551 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05823db2-e266-48d7-84b1-e28225fec84e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:07.516189 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05823db2-e266-48d7-84b1-e28225fec84e None admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:07.516462 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05823db2-e266-48d7-84b1-e28225fec84e None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:07.516992 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05823db2-e266-48d7-84b1-e28225fec84e None admin] PATH_INFO: `/v3/projects/b427ec62a8c545a68a0ce1206bc0787b/users/cce103ff40d04e67b820e5d1b045e14d/roles/3f2ca33902a04b6dbc2ef7d06d7289d3` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:07.518952 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05823db2-e266-48d7-84b1-e28225fec84e None admin] RBAC: Authorizing `identity:create_grant(project_id=b427ec62a8c545a68a0ce1206bc0787b, user_id=cce103ff40d04e67b820e5d1b045e14d, role_id=3f2ca33902a04b6dbc2ef7d06d7289d3)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:07.520366 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05823db2-e266-48d7-84b1-e28225fec84e None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:07.533787 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 547/1084] 10.222.0.49 () {58 vars in 1439 bytes} [Fri Sep 9 12:01:07 2022] PUT /identity/v3/projects/b427ec62a8c545a68a0ce1206bc0787b/users/cce103ff40d04e67b820e5d1b045e14d/roles/3f2ca33902a04b6dbc2ef7d06d7289d3 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:01:08.653626 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedb23c2-f5cd-4d5a-8598-9fad795945d1 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:08.654104 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedb23c2-f5cd-4d5a-8598-9fad795945d1 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:08.654209 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedb23c2-f5cd-4d5a-8598-9fad795945d1 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:08.655524 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 538/1085] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:01:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:01:08.661361 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ec2f8a-ca94-4857-bdfc-3d16b854f4cf None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:08.661880 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ec2f8a-ca94-4857-bdfc-3d16b854f4cf None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:08.661994 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ec2f8a-ca94-4857-bdfc-3d16b854f4cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:08.679706 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-59ec2f8a-ca94-4857-bdfc-3d16b854f4cf None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:01:08.680884 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-59ec2f8a-ca94-4857-bdfc-3d16b854f4cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:01:08.745450 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 548/1086] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:01:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 12:01:09.020917 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedb23c2-f5cd-4d5a-8598-9fad795945d1 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:09.121625 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedb23c2-f5cd-4d5a-8598-9fad795945d1 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:09.149751 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:09.155746 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2137a943-ee4d-4396-9979-16742139c67a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:09.156374 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2137a943-ee4d-4396-9979-16742139c67a None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:09.156675 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2137a943-ee4d-4396-9979-16742139c67a None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:09.156942 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2137a943-ee4d-4396-9979-16742139c67a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:09.167483 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2137a943-ee4d-4396-9979-16742139c67a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:01:09.172248 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-2137a943-ee4d-4396-9979-16742139c67a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 12:01:09.172248 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:01:09.172248 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:01:09.172248 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:01:09.172248 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:01:09.172248 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:01:09.172248 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:01:09.172248 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:01:09.172248 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:01:09.172248 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:01:09.172248 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:01:09.172248 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:01:09.172248 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:01:09.172248 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:01:09.172248 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:01:09.172248 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:01:09.173439 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:01:09.173439 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:01:09.173439 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:01:09.173439 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:01:09.173439 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:01:09.173439 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:01:09.173439 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:01:09.173439 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:01:09.173439 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:01:09.173439 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:01:09.173439 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:01:09.173439 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 12:01:09.173439 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:01:09.173439 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:01:09.173439 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:01:09.173439 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:01:09.174528 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:01:09.174528 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:01:09.174528 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:01:09.174528 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:01:09.174528 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:01:09.174528 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:01:09.174528 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:01:09.174528 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:01:09.174528 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:01:09.174528 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:01:09.174528 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 12:01:09.174528 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:01:09.174528 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 539/1087] 10.222.0.49 () {56 vars in 1120 bytes} [Fri Sep 9 12:01:09 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:01:09.179702 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ec2f8a-ca94-4857-bdfc-3d16b854f4cf None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:09.257697 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:09.277866 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ec2f8a-ca94-4857-bdfc-3d16b854f4cf None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:09.310278 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdc4b6c-16b7-4f8f-b875-8de83d06e50b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:09.310882 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdc4b6c-16b7-4f8f-b875-8de83d06e50b None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:09.311183 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdc4b6c-16b7-4f8f-b875-8de83d06e50b None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:09.311449 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdc4b6c-16b7-4f8f-b875-8de83d06e50b None admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:09.312102 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdc4b6c-16b7-4f8f-b875-8de83d06e50b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:01:09.312398 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdc4b6c-16b7-4f8f-b875-8de83d06e50b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:09.314535 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdc4b6c-16b7-4f8f-b875-8de83d06e50b None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:09.322368 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 549/1088] 10.222.0.49 () {56 vars in 1115 bytes} [Fri Sep 9 12:01:09 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:01:09.328270 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2137a943-ee4d-4396-9979-16742139c67a None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:09.426969 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2137a943-ee4d-4396-9979-16742139c67a None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:09.459450 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d07914a-df58-4511-9554-21bf0b0b9b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:09.460073 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d07914a-df58-4511-9554-21bf0b0b9b1c None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:09.460349 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d07914a-df58-4511-9554-21bf0b0b9b1c None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:09.460618 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d07914a-df58-4511-9554-21bf0b0b9b1c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:09.471242 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d07914a-df58-4511-9554-21bf0b0b9b1c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:09.472677 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d07914a-df58-4511-9554-21bf0b0b9b1c None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:09.486369 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-8d07914a-df58-4511-9554-21bf0b0b9b1c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:01:09.486369 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:01:09.486369 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:01:09.486369 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:01:09.486369 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:01:09.486369 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:01:09.486369 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:01:09.486369 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:01:09.486369 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:01:09.486369 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:01:09.486369 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:01:09.486369 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:01:09.486369 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:01:09.486369 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:01:09.486369 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:01:09.486369 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:01:09.487446 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:01:09.487446 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:01:09.487446 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:01:09.487446 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:01:09.487446 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:01:09.487446 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:01:09.487446 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:01:09.487446 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 12:01:09.487446 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:01:09.487446 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:01:09.487446 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:01:09.487446 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:01:09.487446 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:01:09.487446 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:01:09.487446 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:01:09.487446 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:01:09.487446 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:01:09.488351 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:01:09.488351 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:01:09.488351 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:01:09.488351 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 540/1089] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 12:01:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:01:09.493416 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdc4b6c-16b7-4f8f-b875-8de83d06e50b None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:09.597642 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdc4b6c-16b7-4f8f-b875-8de83d06e50b None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:09.633623 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287932c1-21d7-4a41-9964-7ffbacbec3a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:09.634247 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287932c1-21d7-4a41-9964-7ffbacbec3a2 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:09.634525 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287932c1-21d7-4a41-9964-7ffbacbec3a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:09.634804 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287932c1-21d7-4a41-9964-7ffbacbec3a2 None admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:09.635474 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287932c1-21d7-4a41-9964-7ffbacbec3a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:01:09.635774 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287932c1-21d7-4a41-9964-7ffbacbec3a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:09.637157 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287932c1-21d7-4a41-9964-7ffbacbec3a2 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:09.651000 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 550/1090] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 9 12:01:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:01:09.656659 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d07914a-df58-4511-9554-21bf0b0b9b1c None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:09.681629 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9089a5-c132-4df7-8f6b-a529d9a037f6 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:09.682146 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9089a5-c132-4df7-8f6b-a529d9a037f6 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:09.682146 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9089a5-c132-4df7-8f6b-a529d9a037f6 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:09.683526 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 551/1091] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:01:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 12:01:09.688431 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b641e33a-74a6-49b5-a9be-19ff3a9531b7 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:09.688930 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b641e33a-74a6-49b5-a9be-19ff3a9531b7 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:09.689469 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b641e33a-74a6-49b5-a9be-19ff3a9531b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:09.696520 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:09.702441 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:09.709770 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-b641e33a-74a6-49b5-a9be-19ff3a9531b7 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:01:09.711009 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-b641e33a-74a6-49b5-a9be-19ff3a9531b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:01:09.761197 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d07914a-df58-4511-9554-21bf0b0b9b1c None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:09.779252 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 552/1092] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:01:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 12:01:09.795884 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea6da57-e17a-4d4e-9fd1-35940dd1fb96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:09.796507 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea6da57-e17a-4d4e-9fd1-35940dd1fb96 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:09.796783 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea6da57-e17a-4d4e-9fd1-35940dd1fb96 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:09.797051 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea6da57-e17a-4d4e-9fd1-35940dd1fb96 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:09.804484 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea6da57-e17a-4d4e-9fd1-35940dd1fb96 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:09.805908 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea6da57-e17a-4d4e-9fd1-35940dd1fb96 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:09.817707 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-aea6da57-e17a-4d4e-9fd1-35940dd1fb96 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 12:01:09.817707 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:01:09.817707 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:01:09.817707 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:01:09.817707 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:01:09.817707 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:01:09.817707 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:01:09.817707 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:01:09.817707 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:01:09.817707 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:01:09.817707 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:01:09.817707 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:01:09.817707 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 12:01:09.817707 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 12:01:09.817707 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 12:01:09.817707 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 12:01:09.818795 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:01:09.818795 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:01:09.818795 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 12:01:09.818795 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:01:09.818795 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 12:01:09.818795 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:01:09.818795 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:01:09.818795 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:01:09.818795 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:01:09.818795 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:01:09.818795 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:01:09.818795 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 12:01:09.818795 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:01:09.818795 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:01:09.818795 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:01:09.818795 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:01:09.818795 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:01:09.818795 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:01:09.820237 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:01:09.820237 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:01:09.820237 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 12:01:09.820237 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 12:01:09.820237 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 12:01:09.820237 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 12:01:09.820237 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 12:01:09.820237 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 12:01:09.820237 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 12:01:09.820237 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:01:09.820237 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 541/1093] 10.222.0.49 () {56 vars in 1093 bytes} [Fri Sep 9 12:01:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 12:01:09.824589 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b641e33a-74a6-49b5-a9be-19ff3a9531b7 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:09.921899 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b641e33a-74a6-49b5-a9be-19ff3a9531b7 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:09.950850 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:09.956720 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318c0082-cbac-478a-b028-eabdc4a24894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:09.957347 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318c0082-cbac-478a-b028-eabdc4a24894 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:09.957626 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318c0082-cbac-478a-b028-eabdc4a24894 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:09.957896 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318c0082-cbac-478a-b028-eabdc4a24894 None admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:09.958670 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318c0082-cbac-478a-b028-eabdc4a24894 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:01:09.959006 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318c0082-cbac-478a-b028-eabdc4a24894 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:09.960480 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318c0082-cbac-478a-b028-eabdc4a24894 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:09.977392 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 553/1094] 10.222.0.49 () {56 vars in 1133 bytes} [Fri Sep 9 12:01:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:01:09.983055 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea6da57-e17a-4d4e-9fd1-35940dd1fb96 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:10.051408 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318c0082-cbac-478a-b028-eabdc4a24894 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:10.082993 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea6da57-e17a-4d4e-9fd1-35940dd1fb96 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:10.084533 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:10.118471 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c087cf3c-869c-4611-a951-d79b98d083f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:10.119095 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c087cf3c-869c-4611-a951-d79b98d083f7 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:10.119368 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c087cf3c-869c-4611-a951-d79b98d083f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:10.119638 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c087cf3c-869c-4611-a951-d79b98d083f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:10.150703 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318c0082-cbac-478a-b028-eabdc4a24894 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:10.185104 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748c35eb-0c51-4234-9faa-59271f092688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:10.185658 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748c35eb-0c51-4234-9faa-59271f092688 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:10.185934 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748c35eb-0c51-4234-9faa-59271f092688 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:10.186199 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748c35eb-0c51-4234-9faa-59271f092688 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:10.186524 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c087cf3c-869c-4611-a951-d79b98d083f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:10.188009 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c087cf3c-869c-4611-a951-d79b98d083f7 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:10.197629 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748c35eb-0c51-4234-9faa-59271f092688 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:01:10.202372 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-748c35eb-0c51-4234-9faa-59271f092688 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 12:01:10.202372 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:01:10.202372 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:01:10.202372 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:01:10.202372 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:01:10.202372 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:01:10.202372 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:01:10.202372 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:01:10.202372 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:01:10.202372 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:01:10.202372 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:01:10.202372 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:01:10.202372 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:01:10.202372 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:01:10.202372 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:01:10.202372 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:01:10.203592 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:01:10.203592 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:01:10.203592 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:01:10.203592 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:01:10.203592 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:01:10.203592 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:01:10.203592 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:01:10.203592 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:01:10.203592 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:01:10.203592 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:01:10.203592 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:01:10.203592 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 12:01:10.203592 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:01:10.203592 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:01:10.203592 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:01:10.203592 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:01:10.204438 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:01:10.204438 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:01:10.204438 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:01:10.204438 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:01:10.204438 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:01:10.204438 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:01:10.204438 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:01:10.204438 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:01:10.204438 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:01:10.204438 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:01:10.204438 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 12:01:10.204438 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:01:10.208509 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 554/1095] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Sep 9 12:01:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 12:01:10.209367 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748c35eb-0c51-4234-9faa-59271f092688 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:10.258583 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:10.286496 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 542/1096] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 9 12:01:09 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 12:01:10.292194 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c087cf3c-869c-4611-a951-d79b98d083f7 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:10.294014 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:10.308954 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748c35eb-0c51-4234-9faa-59271f092688 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:10.343370 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e372e7-4d65-4ea0-a25b-464b9b0b24ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:10.343972 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e372e7-4d65-4ea0-a25b-464b9b0b24ca None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:10.344253 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e372e7-4d65-4ea0-a25b-464b9b0b24ca None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:10.344525 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e372e7-4d65-4ea0-a25b-464b9b0b24ca None admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:10.345449 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e372e7-4d65-4ea0-a25b-464b9b0b24ca None admin] RBAC: Adding query filter params (name=admin) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:01:10.345747 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e372e7-4d65-4ea0-a25b-464b9b0b24ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:10.347135 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e372e7-4d65-4ea0-a25b-464b9b0b24ca None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:10.355137 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 555/1097] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 9 12:01:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:01:10.361393 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e372e7-4d65-4ea0-a25b-464b9b0b24ca None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:10.394384 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c087cf3c-869c-4611-a951-d79b98d083f7 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:10.427759 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68560f6-378e-44c4-9f64-6e0ca731b29f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:10.428384 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68560f6-378e-44c4-9f64-6e0ca731b29f None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:10.428666 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68560f6-378e-44c4-9f64-6e0ca731b29f None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:10.428940 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68560f6-378e-44c4-9f64-6e0ca731b29f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:10.439379 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68560f6-378e-44c4-9f64-6e0ca731b29f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:10.440759 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68560f6-378e-44c4-9f64-6e0ca731b29f None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:10.455726 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-a68560f6-378e-44c4-9f64-6e0ca731b29f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:01:10.455726 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:01:10.455726 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:01:10.455726 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:01:10.455726 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:01:10.455726 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:01:10.455726 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:01:10.455726 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:01:10.455726 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:01:10.455726 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:01:10.455726 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:01:10.455726 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:01:10.455726 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:01:10.455726 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:01:10.455726 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:01:10.455726 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:01:10.456861 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:01:10.456861 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:01:10.456861 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:01:10.456861 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:01:10.456861 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:01:10.456861 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:01:10.456861 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:01:10.456861 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 12:01:10.456861 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:01:10.456861 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:01:10.456861 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:01:10.456861 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:01:10.456861 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:01:10.456861 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:01:10.456861 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:01:10.456861 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:01:10.456861 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:01:10.457825 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:01:10.457825 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:01:10.457825 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:01:10.457825 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 543/1098] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 12:01:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:01:10.460591 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e372e7-4d65-4ea0-a25b-464b9b0b24ca None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:10.463711 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68560f6-378e-44c4-9f64-6e0ca731b29f None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:10.474092 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:10.496847 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a4b85f-f61c-4466-8f8f-0fe6810bd760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:10.497469 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a4b85f-f61c-4466-8f8f-0fe6810bd760 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:10.497745 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a4b85f-f61c-4466-8f8f-0fe6810bd760 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:10.498012 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a4b85f-f61c-4466-8f8f-0fe6810bd760 None admin] PATH_INFO: `/v3/users/cce103ff40d04e67b820e5d1b045e14d` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:10.499159 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a4b85f-f61c-4466-8f8f-0fe6810bd760 None admin] RBAC: Authorizing `identity:get_user(user_id=cce103ff40d04e67b820e5d1b045e14d)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:10.500579 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a4b85f-f61c-4466-8f8f-0fe6810bd760 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:10.502400 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 556/1099] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Sep 9 12:01:10 2022] GET /identity/v3/users/cce103ff40d04e67b820e5d1b045e14d => generated 285 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:01:10.508080 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a4b85f-f61c-4466-8f8f-0fe6810bd760 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:10.520857 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:10.568023 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68560f6-378e-44c4-9f64-6e0ca731b29f None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:10.605990 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a4b85f-f61c-4466-8f8f-0fe6810bd760 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:10.607474 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75768ad8-894e-4e64-8cd1-af41a06a3bd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:10.608073 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75768ad8-894e-4e64-8cd1-af41a06a3bd2 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:10.608350 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75768ad8-894e-4e64-8cd1-af41a06a3bd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:10.608638 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75768ad8-894e-4e64-8cd1-af41a06a3bd2 None admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:10.609264 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75768ad8-894e-4e64-8cd1-af41a06a3bd2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:01:10.609558 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75768ad8-894e-4e64-8cd1-af41a06a3bd2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:10.611204 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75768ad8-894e-4e64-8cd1-af41a06a3bd2 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:10.623283 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 544/1100] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 9 12:01:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:01:10.629783 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75768ad8-894e-4e64-8cd1-af41a06a3bd2 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:10.641205 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a55be5-7a71-4fe9-ac0e-f72b8aae37f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:10.642621 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a55be5-7a71-4fe9-ac0e-f72b8aae37f3 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:10.642991 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a55be5-7a71-4fe9-ac0e-f72b8aae37f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:10.643264 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a55be5-7a71-4fe9-ac0e-f72b8aae37f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:10.692207 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:10.711830 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a55be5-7a71-4fe9-ac0e-f72b8aae37f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:10.713265 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a55be5-7a71-4fe9-ac0e-f72b8aae37f3 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:10.732940 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75768ad8-894e-4e64-8cd1-af41a06a3bd2 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:10.747818 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:10.768975 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac1761c-c587-46c2-b14e-097084988c35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:10.769602 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac1761c-c587-46c2-b14e-097084988c35 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:10.769877 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac1761c-c587-46c2-b14e-097084988c35 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:10.770153 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac1761c-c587-46c2-b14e-097084988c35 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:10.776548 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac1761c-c587-46c2-b14e-097084988c35 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:10.777978 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac1761c-c587-46c2-b14e-097084988c35 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:10.789086 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-4ac1761c-c587-46c2-b14e-097084988c35 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:01:10.789086 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:01:10.789086 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:01:10.789086 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:01:10.789086 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:01:10.789086 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:01:10.789086 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:01:10.789086 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:01:10.789086 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:01:10.789086 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:01:10.789086 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:01:10.789086 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:01:10.789086 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 12:01:10.789086 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 12:01:10.789086 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 12:01:10.790188 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 12:01:10.790188 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:01:10.790188 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:01:10.790188 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:01:10.790188 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:01:10.790188 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:01:10.790188 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:01:10.790188 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:01:10.790188 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 12:01:10.790188 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:01:10.790188 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:01:10.790188 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:01:10.790188 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:01:10.790188 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:01:10.790188 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:01:10.790188 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:01:10.790188 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:01:10.796078 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 12:01:10.796078 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 12:01:10.796078 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 12:01:10.796078 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 12:01:10.796078 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 12:01:10.796078 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 12:01:10.796078 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:01:10.796078 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:01:10.796078 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 545/1101] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 9 12:01:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 12:01:10.796078 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac1761c-c587-46c2-b14e-097084988c35 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:10.815606 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 557/1102] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 9 12:01:10 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 09 12:01:10.816601 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a55be5-7a71-4fe9-ac0e-f72b8aae37f3 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:10.895145 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac1761c-c587-46c2-b14e-097084988c35 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:10.895899 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:10.915327 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a55be5-7a71-4fe9-ac0e-f72b8aae37f3 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:10.927808 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca10f77-7a58-4197-a8c4-9d6171deeffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:10.928409 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca10f77-7a58-4197-a8c4-9d6171deeffb None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:10.928683 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca10f77-7a58-4197-a8c4-9d6171deeffb None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:10.928951 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca10f77-7a58-4197-a8c4-9d6171deeffb None admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:10.929631 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca10f77-7a58-4197-a8c4-9d6171deeffb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:01:10.929936 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca10f77-7a58-4197-a8c4-9d6171deeffb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:10.931411 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca10f77-7a58-4197-a8c4-9d6171deeffb None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:10.948252 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 546/1103] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Sep 9 12:01:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:01:10.949226 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca10f77-7a58-4197-a8c4-9d6171deeffb None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:10.949574 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fe787a-4bea-4ede-86b6-2216654da5c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:10.950210 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fe787a-4bea-4ede-86b6-2216654da5c7 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:10.950489 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fe787a-4bea-4ede-86b6-2216654da5c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:10.950759 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fe787a-4bea-4ede-86b6-2216654da5c7 None admin] PATH_INFO: `/v3/projects/b427ec62a8c545a68a0ce1206bc0787b` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:10.951905 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fe787a-4bea-4ede-86b6-2216654da5c7 None admin] RBAC: Authorizing `identity:get_project(project_id=b427ec62a8c545a68a0ce1206bc0787b)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:10.953326 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fe787a-4bea-4ede-86b6-2216654da5c7 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:10.960546 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 558/1104] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Sep 9 12:01:10 2022] GET /identity/v3/projects/b427ec62a8c545a68a0ce1206bc0787b => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:01:10.961336 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fe787a-4bea-4ede-86b6-2216654da5c7 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:10.985023 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:11.051789 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca10f77-7a58-4197-a8c4-9d6171deeffb None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:11.058276 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fe787a-4bea-4ede-86b6-2216654da5c7 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:11.086301 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd22ff47-abfe-48dc-bb3b-e3c7752b7b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:11.086946 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd22ff47-abfe-48dc-bb3b-e3c7752b7b05 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:11.087226 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd22ff47-abfe-48dc-bb3b-e3c7752b7b05 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:11.087498 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd22ff47-abfe-48dc-bb3b-e3c7752b7b05 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:11.088213 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd22ff47-abfe-48dc-bb3b-e3c7752b7b05 None admin] RBAC: Adding query filter params (role.id=db537712f45d4e6980592d3ddefb312a, scope.project.id=fe863fcd75f1406ebbffc5016078f73e, user.id=70bac169d5cd43e1926f310344ac11fa) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:01:11.088526 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd22ff47-abfe-48dc-bb3b-e3c7752b7b05 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:11.089934 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd22ff47-abfe-48dc-bb3b-e3c7752b7b05 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:11.093403 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3217f81-8229-41ab-b01f-9716b7558da4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:11.094000 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3217f81-8229-41ab-b01f-9716b7558da4 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:11.094274 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3217f81-8229-41ab-b01f-9716b7558da4 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:11.094539 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3217f81-8229-41ab-b01f-9716b7558da4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:11.096306 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3217f81-8229-41ab-b01f-9716b7558da4 None admin] RBAC: Adding query filter params (role.id=3f2ca33902a04b6dbc2ef7d06d7289d3, scope.project.id=b427ec62a8c545a68a0ce1206bc0787b, user.id=cce103ff40d04e67b820e5d1b045e14d) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:01:11.096421 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 547/1105] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Sep 9 12:01:10 2022] GET /identity/v3/role_assignments?user.id=70bac169d5cd43e1926f310344ac11fa&scope.project.id=fe863fcd75f1406ebbffc5016078f73e&role.id=db537712f45d4e6980592d3ddefb312a => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:01:11.096619 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3217f81-8229-41ab-b01f-9716b7558da4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:11.098028 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3217f81-8229-41ab-b01f-9716b7558da4 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:11.104066 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 559/1106] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Sep 9 12:01:10 2022] GET /identity/v3/role_assignments?user.id=cce103ff40d04e67b820e5d1b045e14d&scope.project.id=b427ec62a8c545a68a0ce1206bc0787b&role.id=3f2ca33902a04b6dbc2ef7d06d7289d3 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:01:13.326972 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc1a620-cb40-4c13-b079-4341803024b9 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:13.327506 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc1a620-cb40-4c13-b079-4341803024b9 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:13.327506 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc1a620-cb40-4c13-b079-4341803024b9 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:13.328892 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 548/1107] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:01:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:01:13.334255 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5e677e-5cf6-4df5-85b3-63e100590f10 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:13.334679 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5e677e-5cf6-4df5-85b3-63e100590f10 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:13.334806 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5e677e-5cf6-4df5-85b3-63e100590f10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:13.352487 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-ad5e677e-5cf6-4df5-85b3-63e100590f10 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:01:13.353677 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-ad5e677e-5cf6-4df5-85b3-63e100590f10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:01:13.361273 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:13.418386 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 560/1108] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:01:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 12:01:13.691747 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc1a620-cb40-4c13-b079-4341803024b9 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:13.786028 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:13.790730 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc1a620-cb40-4c13-b079-4341803024b9 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:13.824663 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcfd838-4160-4bcd-a7c8-c2c5fb494c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:13.825282 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcfd838-4160-4bcd-a7c8-c2c5fb494c56 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:13.825563 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcfd838-4160-4bcd-a7c8-c2c5fb494c56 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:13.825835 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcfd838-4160-4bcd-a7c8-c2c5fb494c56 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:13.826952 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcfd838-4160-4bcd-a7c8-c2c5fb494c56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:13.828381 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcfd838-4160-4bcd-a7c8-c2c5fb494c56 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:13.830774 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 549/1109] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 12:01:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:01:13.836463 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5e677e-5cf6-4df5-85b3-63e100590f10 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:13.932500 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5e677e-5cf6-4df5-85b3-63e100590f10 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:13.965259 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919c02d1-256e-4edd-8e2e-7fa4ea2aa284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:13.965848 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919c02d1-256e-4edd-8e2e-7fa4ea2aa284 None admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:13.966120 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919c02d1-256e-4edd-8e2e-7fa4ea2aa284 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:13.966386 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919c02d1-256e-4edd-8e2e-7fa4ea2aa284 None admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:13.967054 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919c02d1-256e-4edd-8e2e-7fa4ea2aa284 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:13.968461 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919c02d1-256e-4edd-8e2e-7fa4ea2aa284 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:13.984598 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:13.995918 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 561/1110] 10.222.0.49 () {60 vars in 1132 bytes} [Fri Sep 9 12:01:13 2022] POST /identity/v3/users => generated 286 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:01:16.285024 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3127e591-3459-46d0-b1a3-20c2fb41cc69 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:16.285557 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3127e591-3459-46d0-b1a3-20c2fb41cc69 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:16.285557 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3127e591-3459-46d0-b1a3-20c2fb41cc69 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:16.286924 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 550/1111] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:01:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:01:16.292213 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cafc8d-018a-4cb0-b161-ca66fa3c3560 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:16.292595 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cafc8d-018a-4cb0-b161-ca66fa3c3560 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:16.292754 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cafc8d-018a-4cb0-b161-ca66fa3c3560 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:16.311431 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-89cafc8d-018a-4cb0-b161-ca66fa3c3560 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:01:16.312596 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-89cafc8d-018a-4cb0-b161-ca66fa3c3560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:01:16.382157 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 562/1112] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:01:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 12:01:16.651335 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3127e591-3459-46d0-b1a3-20c2fb41cc69 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:16.753149 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3127e591-3459-46d0-b1a3-20c2fb41cc69 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:16.786888 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bed294-4bef-4788-9a84-53e7d6e2b8c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:16.787528 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bed294-4bef-4788-9a84-53e7d6e2b8c6 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:16.787808 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bed294-4bef-4788-9a84-53e7d6e2b8c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:16.788078 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bed294-4bef-4788-9a84-53e7d6e2b8c6 None admin] PATH_INFO: `/v3/roles/a7c71d2d9914408bb9953f85329d1e80` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:16.789446 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bed294-4bef-4788-9a84-53e7d6e2b8c6 None admin] RBAC: Authorizing `identity:get_role(role_id=a7c71d2d9914408bb9953f85329d1e80)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:16.790809 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bed294-4bef-4788-9a84-53e7d6e2b8c6 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:16.792139 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 551/1113] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Sep 9 12:01:16 2022] GET /identity/v3/roles/a7c71d2d9914408bb9953f85329d1e80 => generated 223 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:01:16.797968 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cafc8d-018a-4cb0-b161-ca66fa3c3560 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:16.893200 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cafc8d-018a-4cb0-b161-ca66fa3c3560 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:16.918145 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:16.926476 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fa04ec-02d0-498b-bb25-e7d6975d6098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:16.927124 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fa04ec-02d0-498b-bb25-e7d6975d6098 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:16.927405 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fa04ec-02d0-498b-bb25-e7d6975d6098 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:16.927675 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fa04ec-02d0-498b-bb25-e7d6975d6098 None admin] PATH_INFO: `/v3/users/c9cb5246291c4a879c55fc73d69eda15` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:16.941106 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fa04ec-02d0-498b-bb25-e7d6975d6098 None admin] RBAC: Authorizing `identity:get_user(user_id=c9cb5246291c4a879c55fc73d69eda15)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:16.942540 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fa04ec-02d0-498b-bb25-e7d6975d6098 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:16.944163 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 563/1114] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Sep 9 12:01:16 2022] GET /identity/v3/users/c9cb5246291c4a879c55fc73d69eda15 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:01:16.949774 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bed294-4bef-4788-9a84-53e7d6e2b8c6 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:16.982459 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:17.047380 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bed294-4bef-4788-9a84-53e7d6e2b8c6 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:17.080491 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68091b5-02cf-448d-adc0-1cb71c5202b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:17.081083 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68091b5-02cf-448d-adc0-1cb71c5202b5 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:17.081355 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68091b5-02cf-448d-adc0-1cb71c5202b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:17.081621 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68091b5-02cf-448d-adc0-1cb71c5202b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:17.146382 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68091b5-02cf-448d-adc0-1cb71c5202b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:17.147846 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68091b5-02cf-448d-adc0-1cb71c5202b5 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:17.179250 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:17.243388 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 552/1115] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 9 12:01:16 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 12:01:17.249442 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fa04ec-02d0-498b-bb25-e7d6975d6098 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:17.345207 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fa04ec-02d0-498b-bb25-e7d6975d6098 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:17.377869 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f339fd-49f0-48e8-b6b9-1b4e071781bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:17.378486 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f339fd-49f0-48e8-b6b9-1b4e071781bc None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:17.378763 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f339fd-49f0-48e8-b6b9-1b4e071781bc None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:17.379067 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f339fd-49f0-48e8-b6b9-1b4e071781bc None admin] PATH_INFO: `/v3/projects/b427ec62a8c545a68a0ce1206bc0787b` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:17.380160 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f339fd-49f0-48e8-b6b9-1b4e071781bc None admin] RBAC: Authorizing `identity:get_project(project_id=b427ec62a8c545a68a0ce1206bc0787b)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:17.381575 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f339fd-49f0-48e8-b6b9-1b4e071781bc None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:17.383203 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 564/1116] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Sep 9 12:01:17 2022] GET /identity/v3/projects/b427ec62a8c545a68a0ce1206bc0787b => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:01:17.389313 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68091b5-02cf-448d-adc0-1cb71c5202b5 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:17.485610 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68091b5-02cf-448d-adc0-1cb71c5202b5 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:17.517773 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b86276-4d83-4dbf-b169-a43fbc97dae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:17.518361 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b86276-4d83-4dbf-b169-a43fbc97dae7 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:17.518637 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b86276-4d83-4dbf-b169-a43fbc97dae7 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:17.518933 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b86276-4d83-4dbf-b169-a43fbc97dae7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:17.519641 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b86276-4d83-4dbf-b169-a43fbc97dae7 None admin] RBAC: Adding query filter params (role.id=a7c71d2d9914408bb9953f85329d1e80, scope.project.id=b427ec62a8c545a68a0ce1206bc0787b, user.id=c9cb5246291c4a879c55fc73d69eda15) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:01:17.519945 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b86276-4d83-4dbf-b169-a43fbc97dae7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:17.521357 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b86276-4d83-4dbf-b169-a43fbc97dae7 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:17.523100 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:17.527232 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 553/1117] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Sep 9 12:01:17 2022] GET /identity/v3/role_assignments?user.id=c9cb5246291c4a879c55fc73d69eda15&scope.project.id=b427ec62a8c545a68a0ce1206bc0787b&role.id=a7c71d2d9914408bb9953f85329d1e80 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:01:19.632437 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96954fe-687d-4cd4-8146-ef34f1c84211 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:19.632967 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96954fe-687d-4cd4-8146-ef34f1c84211 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:19.632967 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96954fe-687d-4cd4-8146-ef34f1c84211 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:19.634299 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 565/1118] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:01:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:01:19.639626 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ef3b19-6e76-4586-90bc-9ede8dae2e94 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:19.640047 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ef3b19-6e76-4586-90bc-9ede8dae2e94 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:19.640207 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ef3b19-6e76-4586-90bc-9ede8dae2e94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:19.659351 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-64ef3b19-6e76-4586-90bc-9ede8dae2e94 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:01:19.660550 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-64ef3b19-6e76-4586-90bc-9ede8dae2e94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:01:19.725704 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 554/1119] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:01:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 12:01:19.995506 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96954fe-687d-4cd4-8146-ef34f1c84211 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:20.049560 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:20.093769 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96954fe-687d-4cd4-8146-ef34f1c84211 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:20.126596 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30837917-8c82-45ed-901b-51bf8aedc8f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:20.127239 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30837917-8c82-45ed-901b-51bf8aedc8f1 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:20.127514 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30837917-8c82-45ed-901b-51bf8aedc8f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:20.127778 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30837917-8c82-45ed-901b-51bf8aedc8f1 None admin] PATH_INFO: `/v3/roles/a7c71d2d9914408bb9953f85329d1e80` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:20.129131 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30837917-8c82-45ed-901b-51bf8aedc8f1 None admin] RBAC: Authorizing `identity:get_role(role_id=a7c71d2d9914408bb9953f85329d1e80)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:20.130477 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30837917-8c82-45ed-901b-51bf8aedc8f1 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:20.131798 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 566/1120] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Sep 9 12:01:19 2022] GET /identity/v3/roles/a7c71d2d9914408bb9953f85329d1e80 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:01:20.137654 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ef3b19-6e76-4586-90bc-9ede8dae2e94 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:20.234279 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ef3b19-6e76-4586-90bc-9ede8dae2e94 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:20.260406 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:20.269064 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c074d9-1b89-4124-ba8a-7c3d316ed12a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:20.269693 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c074d9-1b89-4124-ba8a-7c3d316ed12a None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:20.269972 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c074d9-1b89-4124-ba8a-7c3d316ed12a None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:20.270242 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c074d9-1b89-4124-ba8a-7c3d316ed12a None admin] PATH_INFO: `/v3/users/c9cb5246291c4a879c55fc73d69eda15` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:20.271380 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c074d9-1b89-4124-ba8a-7c3d316ed12a None admin] RBAC: Authorizing `identity:get_user(user_id=c9cb5246291c4a879c55fc73d69eda15)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:20.272807 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c074d9-1b89-4124-ba8a-7c3d316ed12a None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:20.274389 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 555/1121] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Sep 9 12:01:20 2022] GET /identity/v3/users/c9cb5246291c4a879c55fc73d69eda15 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:01:20.280093 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30837917-8c82-45ed-901b-51bf8aedc8f1 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:20.375181 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30837917-8c82-45ed-901b-51bf8aedc8f1 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:20.397289 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:20.408397 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550aa6ce-51a6-436e-a23a-5b92d11b5a99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:20.409006 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550aa6ce-51a6-436e-a23a-5b92d11b5a99 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:20.409281 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550aa6ce-51a6-436e-a23a-5b92d11b5a99 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:20.409547 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550aa6ce-51a6-436e-a23a-5b92d11b5a99 None admin] PATH_INFO: `/v3/projects/b427ec62a8c545a68a0ce1206bc0787b` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:20.410629 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550aa6ce-51a6-436e-a23a-5b92d11b5a99 None admin] RBAC: Authorizing `identity:get_project(project_id=b427ec62a8c545a68a0ce1206bc0787b)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:20.412072 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550aa6ce-51a6-436e-a23a-5b92d11b5a99 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:20.413653 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 567/1122] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Sep 9 12:01:20 2022] GET /identity/v3/projects/b427ec62a8c545a68a0ce1206bc0787b => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:01:20.419727 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c074d9-1b89-4124-ba8a-7c3d316ed12a None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:20.518994 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c074d9-1b89-4124-ba8a-7c3d316ed12a None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:20.552321 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122be942-868e-4736-9b1a-fcc36b6fdd35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:20.552968 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122be942-868e-4736-9b1a-fcc36b6fdd35 None admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:20.553244 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122be942-868e-4736-9b1a-fcc36b6fdd35 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:20.553511 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122be942-868e-4736-9b1a-fcc36b6fdd35 None admin] PATH_INFO: `/v3/projects/b427ec62a8c545a68a0ce1206bc0787b/users/c9cb5246291c4a879c55fc73d69eda15/roles/a7c71d2d9914408bb9953f85329d1e80` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:20.555597 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122be942-868e-4736-9b1a-fcc36b6fdd35 None admin] RBAC: Authorizing `identity:create_grant(project_id=b427ec62a8c545a68a0ce1206bc0787b, user_id=c9cb5246291c4a879c55fc73d69eda15, role_id=a7c71d2d9914408bb9953f85329d1e80)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:20.557037 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122be942-868e-4736-9b1a-fcc36b6fdd35 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:20.570534 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 556/1123] 10.222.0.49 () {58 vars in 1439 bytes} [Fri Sep 9 12:01:20 2022] PUT /identity/v3/projects/b427ec62a8c545a68a0ce1206bc0787b/users/c9cb5246291c4a879c55fc73d69eda15/roles/a7c71d2d9914408bb9953f85329d1e80 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:01:22.693785 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aca4d7-6456-42a5-93a7-53523d211788 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:22.694293 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aca4d7-6456-42a5-93a7-53523d211788 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:22.694386 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aca4d7-6456-42a5-93a7-53523d211788 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:22.695675 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 568/1124] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:01:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:01:22.700912 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a45e19-aec5-4255-a4f0-c00a7c264585 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:22.701279 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a45e19-aec5-4255-a4f0-c00a7c264585 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:22.701452 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a45e19-aec5-4255-a4f0-c00a7c264585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:22.720422 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-75a45e19-aec5-4255-a4f0-c00a7c264585 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:01:22.721593 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-75a45e19-aec5-4255-a4f0-c00a7c264585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:01:22.769694 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:22.787330 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 557/1125] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:01:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 12:01:23.051803 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aca4d7-6456-42a5-93a7-53523d211788 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:23.150891 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aca4d7-6456-42a5-93a7-53523d211788 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:23.184122 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11252ca2-44b5-4b18-a189-c4dd272e3ab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:23.184739 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11252ca2-44b5-4b18-a189-c4dd272e3ab7 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:23.185013 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11252ca2-44b5-4b18-a189-c4dd272e3ab7 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:23.185279 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11252ca2-44b5-4b18-a189-c4dd272e3ab7 None admin] PATH_INFO: `/v3/roles/a7c71d2d9914408bb9953f85329d1e80` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:23.186631 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11252ca2-44b5-4b18-a189-c4dd272e3ab7 None admin] RBAC: Authorizing `identity:get_role(role_id=a7c71d2d9914408bb9953f85329d1e80)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:23.189167 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11252ca2-44b5-4b18-a189-c4dd272e3ab7 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:23.190472 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 569/1126] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Sep 9 12:01:23 2022] GET /identity/v3/roles/a7c71d2d9914408bb9953f85329d1e80 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:01:23.196353 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a45e19-aec5-4255-a4f0-c00a7c264585 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:23.291485 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a45e19-aec5-4255-a4f0-c00a7c264585 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:23.324787 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60f14df-b994-45a9-b8ec-46d915cf3b10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:23.325351 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60f14df-b994-45a9-b8ec-46d915cf3b10 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:23.325628 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60f14df-b994-45a9-b8ec-46d915cf3b10 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:23.325898 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60f14df-b994-45a9-b8ec-46d915cf3b10 None admin] PATH_INFO: `/v3/users/c9cb5246291c4a879c55fc73d69eda15` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:23.327029 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60f14df-b994-45a9-b8ec-46d915cf3b10 None admin] RBAC: Authorizing `identity:get_user(user_id=c9cb5246291c4a879c55fc73d69eda15)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:23.328674 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60f14df-b994-45a9-b8ec-46d915cf3b10 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:23.330229 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 558/1127] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Sep 9 12:01:23 2022] GET /identity/v3/users/c9cb5246291c4a879c55fc73d69eda15 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:01:23.335910 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11252ca2-44b5-4b18-a189-c4dd272e3ab7 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:23.391930 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:23.431651 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11252ca2-44b5-4b18-a189-c4dd272e3ab7 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:23.464149 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc7f448-24d1-4c7a-a1eb-2948c0f5cacb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:23.464739 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc7f448-24d1-4c7a-a1eb-2948c0f5cacb None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:23.465011 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc7f448-24d1-4c7a-a1eb-2948c0f5cacb None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:23.465274 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc7f448-24d1-4c7a-a1eb-2948c0f5cacb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:23.528862 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc7f448-24d1-4c7a-a1eb-2948c0f5cacb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:23.530291 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc7f448-24d1-4c7a-a1eb-2948c0f5cacb None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:23.584668 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:23.628194 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 570/1128] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 9 12:01:23 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 12:01:23.634518 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60f14df-b994-45a9-b8ec-46d915cf3b10 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:23.691389 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:23.734699 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60f14df-b994-45a9-b8ec-46d915cf3b10 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:23.768725 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7f3693-b1bf-4f08-8e79-bb72d4c60a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:23.769348 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7f3693-b1bf-4f08-8e79-bb72d4c60a73 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:23.769627 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7f3693-b1bf-4f08-8e79-bb72d4c60a73 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:23.769897 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7f3693-b1bf-4f08-8e79-bb72d4c60a73 None admin] PATH_INFO: `/v3/projects/b427ec62a8c545a68a0ce1206bc0787b` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:23.771048 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7f3693-b1bf-4f08-8e79-bb72d4c60a73 None admin] RBAC: Authorizing `identity:get_project(project_id=b427ec62a8c545a68a0ce1206bc0787b)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:23.772485 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7f3693-b1bf-4f08-8e79-bb72d4c60a73 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:23.774099 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 559/1129] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Sep 9 12:01:23 2022] GET /identity/v3/projects/b427ec62a8c545a68a0ce1206bc0787b => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:01:23.780417 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc7f448-24d1-4c7a-a1eb-2948c0f5cacb None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:23.875996 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc7f448-24d1-4c7a-a1eb-2948c0f5cacb None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:23.908411 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b5047f-3762-4071-bbed-6554573ce891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:23.909148 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b5047f-3762-4071-bbed-6554573ce891 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:23.909426 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b5047f-3762-4071-bbed-6554573ce891 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:23.909774 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b5047f-3762-4071-bbed-6554573ce891 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:23.910489 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b5047f-3762-4071-bbed-6554573ce891 None admin] RBAC: Adding query filter params (role.id=a7c71d2d9914408bb9953f85329d1e80, scope.project.id=b427ec62a8c545a68a0ce1206bc0787b, user.id=c9cb5246291c4a879c55fc73d69eda15) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:01:23.910798 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b5047f-3762-4071-bbed-6554573ce891 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:23.912250 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b5047f-3762-4071-bbed-6554573ce891 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:23.917762 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 571/1130] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Sep 9 12:01:23 2022] GET /identity/v3/role_assignments?user.id=c9cb5246291c4a879c55fc73d69eda15&scope.project.id=b427ec62a8c545a68a0ce1206bc0787b&role.id=a7c71d2d9914408bb9953f85329d1e80 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:01:26.071046 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0c97db-d42f-44b7-bd4b-33111fc65dc6 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:26.071563 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0c97db-d42f-44b7-bd4b-33111fc65dc6 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:26.071563 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0c97db-d42f-44b7-bd4b-33111fc65dc6 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:26.072930 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 560/1131] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:01:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:01:26.079404 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706660d1-0d92-4389-83dc-aaabfb816452 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:26.079861 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706660d1-0d92-4389-83dc-aaabfb816452 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:26.080077 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706660d1-0d92-4389-83dc-aaabfb816452 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:26.098096 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-706660d1-0d92-4389-83dc-aaabfb816452 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:01:26.099309 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-706660d1-0d92-4389-83dc-aaabfb816452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:01:26.115100 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:26.164455 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 572/1132] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:01:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 12:01:26.434825 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0c97db-d42f-44b7-bd4b-33111fc65dc6 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:26.534498 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0c97db-d42f-44b7-bd4b-33111fc65dc6 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:26.556608 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:26.569160 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee75f8ed-105d-426b-b69d-9e6e649981a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:26.569766 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee75f8ed-105d-426b-b69d-9e6e649981a5 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:26.570041 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee75f8ed-105d-426b-b69d-9e6e649981a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:26.570302 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee75f8ed-105d-426b-b69d-9e6e649981a5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:26.571409 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee75f8ed-105d-426b-b69d-9e6e649981a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:26.572819 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee75f8ed-105d-426b-b69d-9e6e649981a5 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:26.574358 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 561/1133] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 12:01:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:01:26.580392 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706660d1-0d92-4389-83dc-aaabfb816452 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:26.676174 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706660d1-0d92-4389-83dc-aaabfb816452 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:26.709005 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7378b8f0-be6b-4d99-a4fe-43853530f271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:26.709536 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7378b8f0-be6b-4d99-a4fe-43853530f271 None admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:26.709815 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7378b8f0-be6b-4d99-a4fe-43853530f271 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:26.710083 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7378b8f0-be6b-4d99-a4fe-43853530f271 None admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:26.710728 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7378b8f0-be6b-4d99-a4fe-43853530f271 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:26.712175 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7378b8f0-be6b-4d99-a4fe-43853530f271 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:26.738448 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 573/1134] 10.222.0.49 () {60 vars in 1141 bytes} [Fri Sep 9 12:01:26 2022] POST /identity/v3/projects => generated 311 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:01:28.909737 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6abcfe0-29f6-470e-8dc0-b4d604d24d79 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:28.910289 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6abcfe0-29f6-470e-8dc0-b4d604d24d79 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:28.910289 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6abcfe0-29f6-470e-8dc0-b4d604d24d79 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:28.911568 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 562/1135] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:01:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:01:28.916696 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d7bd9b-d2d6-441d-9830-17cc326a2d13 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:28.917123 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d7bd9b-d2d6-441d-9830-17cc326a2d13 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:28.917222 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d7bd9b-d2d6-441d-9830-17cc326a2d13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:28.924621 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:28.936681 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-20d7bd9b-d2d6-441d-9830-17cc326a2d13 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:01:28.938512 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-20d7bd9b-d2d6-441d-9830-17cc326a2d13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:01:29.002427 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 574/1136] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:01:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 12:01:29.273044 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6abcfe0-29f6-470e-8dc0-b4d604d24d79 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:29.372663 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6abcfe0-29f6-470e-8dc0-b4d604d24d79 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:29.405290 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cbc319-2e46-4c7a-92c3-ff8518a8f8aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:29.405905 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cbc319-2e46-4c7a-92c3-ff8518a8f8aa None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:29.406742 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cbc319-2e46-4c7a-92c3-ff8518a8f8aa None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:29.407046 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cbc319-2e46-4c7a-92c3-ff8518a8f8aa None admin] PATH_INFO: `/v3/domains/default` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:29.408136 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cbc319-2e46-4c7a-92c3-ff8518a8f8aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:29.409565 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cbc319-2e46-4c7a-92c3-ff8518a8f8aa None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:29.411142 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 563/1137] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 12:01:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:01:29.416713 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d7bd9b-d2d6-441d-9830-17cc326a2d13 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:29.512109 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d7bd9b-d2d6-441d-9830-17cc326a2d13 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:29.544199 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c884a1f-9913-4c51-8fcf-90a76d8b3905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:29.544725 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c884a1f-9913-4c51-8fcf-90a76d8b3905 None admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:29.544997 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c884a1f-9913-4c51-8fcf-90a76d8b3905 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:29.545263 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c884a1f-9913-4c51-8fcf-90a76d8b3905 None admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:29.545899 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c884a1f-9913-4c51-8fcf-90a76d8b3905 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:29.548212 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c884a1f-9913-4c51-8fcf-90a76d8b3905 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:29.555976 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:29.569389 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 575/1138] 10.222.0.49 () {60 vars in 1132 bytes} [Fri Sep 9 12:01:29 2022] POST /identity/v3/users => generated 286 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:01:31.756012 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfc019a-4dca-4b7c-bbc7-3ba0e0533d85 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:31.756548 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfc019a-4dca-4b7c-bbc7-3ba0e0533d85 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:31.756548 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfc019a-4dca-4b7c-bbc7-3ba0e0533d85 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:31.757912 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 564/1139] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:01:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:01:31.763089 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598ead2d-8447-4bae-bad2-21bef27eea5a None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:31.763524 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598ead2d-8447-4bae-bad2-21bef27eea5a None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:31.763630 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598ead2d-8447-4bae-bad2-21bef27eea5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:31.781523 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-598ead2d-8447-4bae-bad2-21bef27eea5a None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:01:31.782704 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-598ead2d-8447-4bae-bad2-21bef27eea5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:01:31.847229 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 576/1140] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:01:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 12:01:32.120233 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfc019a-4dca-4b7c-bbc7-3ba0e0533d85 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:32.178971 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:32.221734 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfc019a-4dca-4b7c-bbc7-3ba0e0533d85 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:32.254658 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c66a319-ffc0-491c-b05e-02fa29e94076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:32.255294 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c66a319-ffc0-491c-b05e-02fa29e94076 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:32.255566 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c66a319-ffc0-491c-b05e-02fa29e94076 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:32.255828 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c66a319-ffc0-491c-b05e-02fa29e94076 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:32.266115 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c66a319-ffc0-491c-b05e-02fa29e94076 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:01:32.270982 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-0c66a319-ffc0-491c-b05e-02fa29e94076 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 12:01:32.270982 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:01:32.270982 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:01:32.270982 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:01:32.270982 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:01:32.270982 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:01:32.270982 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:01:32.270982 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:01:32.270982 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:01:32.270982 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:01:32.270982 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:01:32.270982 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:01:32.270982 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:01:32.270982 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:01:32.270982 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:01:32.270982 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:01:32.272171 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:01:32.272171 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:01:32.272171 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:01:32.272171 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:01:32.272171 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:01:32.272171 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:01:32.272171 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:01:32.272171 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:01:32.272171 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:01:32.272171 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:01:32.272171 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:01:32.272171 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 12:01:32.272171 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:01:32.272171 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:01:32.272171 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:01:32.272171 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:01:32.273079 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:01:32.273079 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:01:32.273079 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:01:32.273079 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:01:32.273079 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:01:32.273079 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:01:32.273079 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:01:32.273079 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:01:32.273079 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:01:32.273079 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:01:32.273079 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 12:01:32.273079 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:01:32.273079 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 565/1141] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Sep 9 12:01:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:01:32.278072 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598ead2d-8447-4bae-bad2-21bef27eea5a None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:32.372367 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598ead2d-8447-4bae-bad2-21bef27eea5a None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:32.391630 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:32.406263 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bee48d5-973f-4823-9f9d-ca88dc0765cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:32.406929 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bee48d5-973f-4823-9f9d-ca88dc0765cd None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:32.407215 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bee48d5-973f-4823-9f9d-ca88dc0765cd None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:32.407481 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bee48d5-973f-4823-9f9d-ca88dc0765cd None admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:32.408134 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bee48d5-973f-4823-9f9d-ca88dc0765cd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:01:32.408427 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bee48d5-973f-4823-9f9d-ca88dc0765cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:32.409767 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bee48d5-973f-4823-9f9d-ca88dc0765cd None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:32.417405 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 577/1142] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 9 12:01:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:01:32.423837 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c66a319-ffc0-491c-b05e-02fa29e94076 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:32.521666 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c66a319-ffc0-491c-b05e-02fa29e94076 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:32.538217 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:32.555352 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05942439-bc00-4f7e-b5e4-e834d7eb028d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:32.555965 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05942439-bc00-4f7e-b5e4-e834d7eb028d None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:32.556237 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05942439-bc00-4f7e-b5e4-e834d7eb028d None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:32.556508 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05942439-bc00-4f7e-b5e4-e834d7eb028d None admin] PATH_INFO: `/v3/users/3a3eaaaac3964668acecec7bafa78d7d` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:32.570018 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05942439-bc00-4f7e-b5e4-e834d7eb028d None admin] RBAC: Authorizing `identity:get_user(user_id=3a3eaaaac3964668acecec7bafa78d7d)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:32.571466 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05942439-bc00-4f7e-b5e4-e834d7eb028d None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:32.573037 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 566/1143] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Sep 9 12:01:32 2022] GET /identity/v3/users/3a3eaaaac3964668acecec7bafa78d7d => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:01:32.579328 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bee48d5-973f-4823-9f9d-ca88dc0765cd None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:32.674466 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bee48d5-973f-4823-9f9d-ca88dc0765cd None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:32.706549 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222b291c-cde6-486c-a0e2-0e5c95ffe059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:32.707167 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222b291c-cde6-486c-a0e2-0e5c95ffe059 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:32.707442 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222b291c-cde6-486c-a0e2-0e5c95ffe059 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:32.707708 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222b291c-cde6-486c-a0e2-0e5c95ffe059 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:32.754126 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:32.772540 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222b291c-cde6-486c-a0e2-0e5c95ffe059 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:32.773968 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222b291c-cde6-486c-a0e2-0e5c95ffe059 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:32.868258 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 578/1144] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 9 12:01:32 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 12:01:32.874232 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05942439-bc00-4f7e-b5e4-e834d7eb028d None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:32.969959 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05942439-bc00-4f7e-b5e4-e834d7eb028d None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:33.002223 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a27577-7e96-4999-b31c-9850e7cbf42e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:33.002830 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a27577-7e96-4999-b31c-9850e7cbf42e None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:33.003130 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a27577-7e96-4999-b31c-9850e7cbf42e None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:33.003394 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a27577-7e96-4999-b31c-9850e7cbf42e None admin] PATH_INFO: `/v3/projects/5eea347b2fdb41a69362258937213fbc` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:33.004485 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a27577-7e96-4999-b31c-9850e7cbf42e None admin] RBAC: Authorizing `identity:get_project(project_id=5eea347b2fdb41a69362258937213fbc)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:33.005894 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a27577-7e96-4999-b31c-9850e7cbf42e None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:33.007510 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 567/1145] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Sep 9 12:01:32 2022] GET /identity/v3/projects/5eea347b2fdb41a69362258937213fbc => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:01:33.013535 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222b291c-cde6-486c-a0e2-0e5c95ffe059 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:33.091012 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:33.108641 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222b291c-cde6-486c-a0e2-0e5c95ffe059 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:33.141874 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5834227d-bcaa-41c8-bca2-e27216500598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:33.142466 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5834227d-bcaa-41c8-bca2-e27216500598 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:33.142742 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5834227d-bcaa-41c8-bca2-e27216500598 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:33.143042 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5834227d-bcaa-41c8-bca2-e27216500598 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:33.143756 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5834227d-bcaa-41c8-bca2-e27216500598 None admin] RBAC: Adding query filter params (role.id=3f2ca33902a04b6dbc2ef7d06d7289d3, scope.project.id=5eea347b2fdb41a69362258937213fbc, user.id=3a3eaaaac3964668acecec7bafa78d7d) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:01:33.144059 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5834227d-bcaa-41c8-bca2-e27216500598 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:33.145473 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5834227d-bcaa-41c8-bca2-e27216500598 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:33.150630 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 579/1146] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Sep 9 12:01:33 2022] GET /identity/v3/role_assignments?user.id=3a3eaaaac3964668acecec7bafa78d7d&scope.project.id=5eea347b2fdb41a69362258937213fbc&role.id=3f2ca33902a04b6dbc2ef7d06d7289d3 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:01:35.276340 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7624dff-782e-4af8-bcf3-7debc53125df None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:35.276910 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7624dff-782e-4af8-bcf3-7debc53125df None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:35.276910 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7624dff-782e-4af8-bcf3-7debc53125df None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:35.278236 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 568/1147] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:01:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:01:35.284259 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d732ed7-5f38-4417-9f05-9fa610433368 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:35.284675 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d732ed7-5f38-4417-9f05-9fa610433368 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:35.284794 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d732ed7-5f38-4417-9f05-9fa610433368 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:35.303427 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-8d732ed7-5f38-4417-9f05-9fa610433368 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:01:35.304611 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-8d732ed7-5f38-4417-9f05-9fa610433368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:01:35.367880 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 580/1148] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:01:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 12:01:35.634015 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7624dff-782e-4af8-bcf3-7debc53125df None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:35.681179 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:35.735885 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7624dff-782e-4af8-bcf3-7debc53125df None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:35.768186 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad109be7-51d8-4fbd-94bb-182bbe4756db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:35.768804 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad109be7-51d8-4fbd-94bb-182bbe4756db None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:35.769087 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad109be7-51d8-4fbd-94bb-182bbe4756db None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:35.769357 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad109be7-51d8-4fbd-94bb-182bbe4756db None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:35.779070 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad109be7-51d8-4fbd-94bb-182bbe4756db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:01:35.784445 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-ad109be7-51d8-4fbd-94bb-182bbe4756db None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 12:01:35.784445 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:01:35.784445 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:01:35.784445 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:01:35.784445 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:01:35.784445 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:01:35.784445 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:01:35.784445 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:01:35.784445 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:01:35.784445 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:01:35.784445 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:01:35.784445 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:01:35.784445 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:01:35.784445 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:01:35.784445 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:01:35.784445 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:01:35.785377 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:01:35.785377 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:01:35.785377 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:01:35.785377 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:01:35.785377 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:01:35.785377 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:01:35.785377 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:01:35.785377 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:01:35.785377 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:01:35.785377 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:01:35.785377 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:01:35.785377 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 12:01:35.785377 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:01:35.785377 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:01:35.785377 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:01:35.785377 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:01:35.786313 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:01:35.786313 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:01:35.786313 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:01:35.786313 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:01:35.786313 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:01:35.786313 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:01:35.786313 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:01:35.786313 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:01:35.786313 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:01:35.786313 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:01:35.786313 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 12:01:35.786313 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:01:35.786313 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 569/1149] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Sep 9 12:01:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:01:35.791720 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d732ed7-5f38-4417-9f05-9fa610433368 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:35.868283 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:35.890662 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d732ed7-5f38-4417-9f05-9fa610433368 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:35.927604 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac863160-7272-490d-a35d-784f4d24155a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:35.928198 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac863160-7272-490d-a35d-784f4d24155a None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:35.928479 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac863160-7272-490d-a35d-784f4d24155a None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:35.928749 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac863160-7272-490d-a35d-784f4d24155a None admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:35.929400 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac863160-7272-490d-a35d-784f4d24155a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:01:35.929698 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac863160-7272-490d-a35d-784f4d24155a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:35.931080 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac863160-7272-490d-a35d-784f4d24155a None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:35.939048 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 581/1150] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 9 12:01:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:01:35.944868 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad109be7-51d8-4fbd-94bb-182bbe4756db None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:36.042437 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad109be7-51d8-4fbd-94bb-182bbe4756db None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:36.044750 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:36.077129 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7daeb0-a1c9-4ecd-8c96-8309993e3d9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:36.078266 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7daeb0-a1c9-4ecd-8c96-8309993e3d9f None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:36.078653 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7daeb0-a1c9-4ecd-8c96-8309993e3d9f None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:36.079020 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7daeb0-a1c9-4ecd-8c96-8309993e3d9f None admin] PATH_INFO: `/v3/users/3a3eaaaac3964668acecec7bafa78d7d` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:36.080401 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7daeb0-a1c9-4ecd-8c96-8309993e3d9f None admin] RBAC: Authorizing `identity:get_user(user_id=3a3eaaaac3964668acecec7bafa78d7d)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:36.082123 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7daeb0-a1c9-4ecd-8c96-8309993e3d9f None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:36.083789 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 570/1151] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Sep 9 12:01:35 2022] GET /identity/v3/users/3a3eaaaac3964668acecec7bafa78d7d => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:01:36.089527 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac863160-7272-490d-a35d-784f4d24155a None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:36.185907 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac863160-7272-490d-a35d-784f4d24155a None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:36.219588 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d268919-843d-4c89-8ed2-30e13a05937e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:36.220318 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d268919-843d-4c89-8ed2-30e13a05937e None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:36.220597 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d268919-843d-4c89-8ed2-30e13a05937e None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:36.220865 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d268919-843d-4c89-8ed2-30e13a05937e None admin] PATH_INFO: `/v3/projects/5eea347b2fdb41a69362258937213fbc` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:36.221970 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d268919-843d-4c89-8ed2-30e13a05937e None admin] RBAC: Authorizing `identity:get_project(project_id=5eea347b2fdb41a69362258937213fbc)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:36.224400 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d268919-843d-4c89-8ed2-30e13a05937e None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:36.226787 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 582/1152] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Sep 9 12:01:36 2022] GET /identity/v3/projects/5eea347b2fdb41a69362258937213fbc => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:01:36.232526 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7daeb0-a1c9-4ecd-8c96-8309993e3d9f None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:36.328863 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7daeb0-a1c9-4ecd-8c96-8309993e3d9f None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:36.361673 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507d55cf-c2a7-4205-96f5-109b03ea9663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:36.362305 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507d55cf-c2a7-4205-96f5-109b03ea9663 None admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:36.362578 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507d55cf-c2a7-4205-96f5-109b03ea9663 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:36.362841 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507d55cf-c2a7-4205-96f5-109b03ea9663 None admin] PATH_INFO: `/v3/projects/5eea347b2fdb41a69362258937213fbc/users/3a3eaaaac3964668acecec7bafa78d7d/roles/3f2ca33902a04b6dbc2ef7d06d7289d3` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:36.364723 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507d55cf-c2a7-4205-96f5-109b03ea9663 None admin] RBAC: Authorizing `identity:create_grant(project_id=5eea347b2fdb41a69362258937213fbc, user_id=3a3eaaaac3964668acecec7bafa78d7d, role_id=3f2ca33902a04b6dbc2ef7d06d7289d3)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:36.366124 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507d55cf-c2a7-4205-96f5-109b03ea9663 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:36.378693 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 571/1153] 10.222.0.49 () {58 vars in 1439 bytes} [Fri Sep 9 12:01:36 2022] PUT /identity/v3/projects/5eea347b2fdb41a69362258937213fbc/users/3a3eaaaac3964668acecec7bafa78d7d/roles/3f2ca33902a04b6dbc2ef7d06d7289d3 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:01:38.522090 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c398efc-0603-4867-9839-aaff43189ecf None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:38.522734 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c398efc-0603-4867-9839-aaff43189ecf None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:38.522734 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c398efc-0603-4867-9839-aaff43189ecf None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:38.524028 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 583/1154] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:01:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:01:38.529395 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a7776b-23c3-4868-a61c-c5b09cb4c999 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:38.529793 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a7776b-23c3-4868-a61c-c5b09cb4c999 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:38.529936 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a7776b-23c3-4868-a61c-c5b09cb4c999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:38.548442 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-57a7776b-23c3-4868-a61c-c5b09cb4c999 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:01:38.549617 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-57a7776b-23c3-4868-a61c-c5b09cb4c999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:01:38.574627 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:38.616704 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 572/1155] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:01:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 12:01:38.890679 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c398efc-0603-4867-9839-aaff43189ecf None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:38.899819 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:38.989849 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c398efc-0603-4867-9839-aaff43189ecf None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:39.021622 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c405a7e-e737-41dd-93ff-d31f0cbe304a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:39.022238 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c405a7e-e737-41dd-93ff-d31f0cbe304a None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:39.022517 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c405a7e-e737-41dd-93ff-d31f0cbe304a None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:39.022774 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c405a7e-e737-41dd-93ff-d31f0cbe304a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:39.033546 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c405a7e-e737-41dd-93ff-d31f0cbe304a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:01:39.038220 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-4c405a7e-e737-41dd-93ff-d31f0cbe304a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 12:01:39.038220 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:01:39.038220 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:01:39.038220 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:01:39.038220 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:01:39.038220 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:01:39.038220 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:01:39.038220 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:01:39.038220 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:01:39.038220 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:01:39.038220 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:01:39.038220 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:01:39.038220 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:01:39.038220 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:01:39.038220 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:01:39.038220 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:01:39.039434 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:01:39.039434 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:01:39.039434 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:01:39.039434 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:01:39.039434 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:01:39.039434 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:01:39.039434 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:01:39.039434 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:01:39.039434 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:01:39.039434 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:01:39.039434 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:01:39.039434 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 12:01:39.039434 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:01:39.039434 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:01:39.039434 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:01:39.039434 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:01:39.040737 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:01:39.040737 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:01:39.040737 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:01:39.040737 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:01:39.040737 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:01:39.040737 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:01:39.040737 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:01:39.040737 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:01:39.040737 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:01:39.040737 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:01:39.040737 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 12:01:39.040737 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:01:39.040737 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 584/1156] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Sep 9 12:01:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:01:39.045259 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a7776b-23c3-4868-a61c-c5b09cb4c999 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:39.143140 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a7776b-23c3-4868-a61c-c5b09cb4c999 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:39.176047 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f698092c-3ba9-43a9-b711-a43780f39ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:39.176649 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f698092c-3ba9-43a9-b711-a43780f39ac7 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:39.176927 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f698092c-3ba9-43a9-b711-a43780f39ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:39.177196 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f698092c-3ba9-43a9-b711-a43780f39ac7 None admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:39.177847 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f698092c-3ba9-43a9-b711-a43780f39ac7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:01:39.178147 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f698092c-3ba9-43a9-b711-a43780f39ac7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:39.179535 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f698092c-3ba9-43a9-b711-a43780f39ac7 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:39.188168 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 573/1157] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 9 12:01:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:01:39.194356 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c405a7e-e737-41dd-93ff-d31f0cbe304a None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:39.259279 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:39.292714 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c405a7e-e737-41dd-93ff-d31f0cbe304a None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:39.325539 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482d0a57-d87f-42ef-b7e0-8f216c3ba831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:39.326144 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482d0a57-d87f-42ef-b7e0-8f216c3ba831 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:39.326415 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482d0a57-d87f-42ef-b7e0-8f216c3ba831 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:39.326679 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482d0a57-d87f-42ef-b7e0-8f216c3ba831 None admin] PATH_INFO: `/v3/users/3a3eaaaac3964668acecec7bafa78d7d` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:39.328061 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482d0a57-d87f-42ef-b7e0-8f216c3ba831 None admin] RBAC: Authorizing `identity:get_user(user_id=3a3eaaaac3964668acecec7bafa78d7d)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:39.329472 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482d0a57-d87f-42ef-b7e0-8f216c3ba831 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:39.331034 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 585/1158] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Sep 9 12:01:39 2022] GET /identity/v3/users/3a3eaaaac3964668acecec7bafa78d7d => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:01:39.336748 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f698092c-3ba9-43a9-b711-a43780f39ac7 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:39.362219 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:39.434240 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f698092c-3ba9-43a9-b711-a43780f39ac7 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:39.466838 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce5b738-663c-4496-8da9-99e1153a68cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:39.467457 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce5b738-663c-4496-8da9-99e1153a68cf None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:39.467738 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce5b738-663c-4496-8da9-99e1153a68cf None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:39.468498 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce5b738-663c-4496-8da9-99e1153a68cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:39.533912 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce5b738-663c-4496-8da9-99e1153a68cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:39.535386 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce5b738-663c-4496-8da9-99e1153a68cf None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:39.558558 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:39.631792 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 574/1159] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 9 12:01:39 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 12:01:39.637588 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482d0a57-d87f-42ef-b7e0-8f216c3ba831 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:39.734429 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482d0a57-d87f-42ef-b7e0-8f216c3ba831 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:39.765720 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d62203c-482a-4a90-9307-4ca0102175a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:39.766333 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d62203c-482a-4a90-9307-4ca0102175a7 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:39.766702 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d62203c-482a-4a90-9307-4ca0102175a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:39.767006 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d62203c-482a-4a90-9307-4ca0102175a7 None admin] PATH_INFO: `/v3/projects/5eea347b2fdb41a69362258937213fbc` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:39.768136 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d62203c-482a-4a90-9307-4ca0102175a7 None admin] RBAC: Authorizing `identity:get_project(project_id=5eea347b2fdb41a69362258937213fbc)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:39.769557 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d62203c-482a-4a90-9307-4ca0102175a7 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:39.771188 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 586/1160] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Sep 9 12:01:39 2022] GET /identity/v3/projects/5eea347b2fdb41a69362258937213fbc => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:01:39.777207 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce5b738-663c-4496-8da9-99e1153a68cf None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:39.874167 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce5b738-663c-4496-8da9-99e1153a68cf None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:39.897323 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:39.908927 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4366783f-2f9b-483b-ad46-4acd1cee4854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:39.909517 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4366783f-2f9b-483b-ad46-4acd1cee4854 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:39.909790 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4366783f-2f9b-483b-ad46-4acd1cee4854 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:39.910054 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4366783f-2f9b-483b-ad46-4acd1cee4854 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:39.910764 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4366783f-2f9b-483b-ad46-4acd1cee4854 None admin] RBAC: Adding query filter params (role.id=3f2ca33902a04b6dbc2ef7d06d7289d3, scope.project.id=5eea347b2fdb41a69362258937213fbc, user.id=3a3eaaaac3964668acecec7bafa78d7d) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:01:39.911093 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4366783f-2f9b-483b-ad46-4acd1cee4854 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:39.912503 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4366783f-2f9b-483b-ad46-4acd1cee4854 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:39.918136 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 575/1161] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Sep 9 12:01:39 2022] GET /identity/v3/role_assignments?user.id=3a3eaaaac3964668acecec7bafa78d7d&scope.project.id=5eea347b2fdb41a69362258937213fbc&role.id=3f2ca33902a04b6dbc2ef7d06d7289d3 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:01:42.051978 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24c7ad5-7a8b-416c-850d-15f3c69566da None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:42.052522 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24c7ad5-7a8b-416c-850d-15f3c69566da None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:42.052522 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24c7ad5-7a8b-416c-850d-15f3c69566da None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:42.053833 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 587/1162] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:01:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:01:42.058959 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22438f5-4097-4bdd-9b6f-8ae4e9e51f78 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:42.059353 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22438f5-4097-4bdd-9b6f-8ae4e9e51f78 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:42.059554 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22438f5-4097-4bdd-9b6f-8ae4e9e51f78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:42.079033 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-b22438f5-4097-4bdd-9b6f-8ae4e9e51f78 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:01:42.080208 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-b22438f5-4097-4bdd-9b6f-8ae4e9e51f78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:01:42.144221 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 576/1163] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:01:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 12:01:42.410117 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24c7ad5-7a8b-416c-850d-15f3c69566da None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:42.412899 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:42.507477 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24c7ad5-7a8b-416c-850d-15f3c69566da None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:42.539373 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d45a42-3a92-4ab1-b628-8509155858d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:42.539963 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d45a42-3a92-4ab1-b628-8509155858d6 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:42.540236 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d45a42-3a92-4ab1-b628-8509155858d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:42.540499 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d45a42-3a92-4ab1-b628-8509155858d6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:42.605008 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d45a42-3a92-4ab1-b628-8509155858d6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:42.606430 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d45a42-3a92-4ab1-b628-8509155858d6 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:42.608159 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:42.702876 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 588/1164] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 9 12:01:42 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 12:01:42.708722 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22438f5-4097-4bdd-9b6f-8ae4e9e51f78 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:42.803475 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22438f5-4097-4bdd-9b6f-8ae4e9e51f78 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:42.820840 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:42.837960 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04d9947-4497-4812-8fc8-a395760ded8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:42.838575 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04d9947-4497-4812-8fc8-a395760ded8f None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:42.838854 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04d9947-4497-4812-8fc8-a395760ded8f None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:42.839153 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04d9947-4497-4812-8fc8-a395760ded8f None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:42.845644 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04d9947-4497-4812-8fc8-a395760ded8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:42.847111 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04d9947-4497-4812-8fc8-a395760ded8f None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:42.858118 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-d04d9947-4497-4812-8fc8-a395760ded8f None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 09 12:01:42.858118 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:01:42.858118 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 09 12:01:42.858118 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 09 12:01:42.858118 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 12:01:42.858118 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 12:01:42.858118 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 12:01:42.858118 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 12:01:42.858118 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 09 12:01:42.858118 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:01:42.858118 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 12:01:42.858118 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:01:42.858118 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:01:42.858118 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:01:42.858118 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:01:42.858118 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:01:42.858118 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:01:42.860224 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:01:42.860224 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:01:42.860224 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:01:42.860224 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:01:42.860224 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:01:42.860224 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:01:42.860224 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:01:42.860224 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:01:42.860224 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:01:42.860224 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:01:42.860224 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:01:42.860224 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:01:42.860224 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:01:42.860224 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:01:42.860224 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:01:42.860224 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:01:42.861264 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:01:42.861264 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 12:01:42.861264 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:01:42.861264 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:01:42.861264 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:01:42.861264 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:01:42.861264 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:01:42.861264 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:01:42.861264 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:01:42.861264 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:01:42.861264 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 09 12:01:42.861264 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:01:42.861264 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 09 12:01:42.861264 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:01:42.861264 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 577/1165] 10.222.0.49 () {56 vars in 1117 bytes} [Fri Sep 9 12:01:42 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:01:42.866122 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d45a42-3a92-4ab1-b628-8509155858d6 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:42.961128 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d45a42-3a92-4ab1-b628-8509155858d6 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:42.962549 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:42.995511 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f76d0f6-8773-485d-b07b-9d3697a9ca48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:42.996107 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f76d0f6-8773-485d-b07b-9d3697a9ca48 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:42.996385 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f76d0f6-8773-485d-b07b-9d3697a9ca48 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:42.996657 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f76d0f6-8773-485d-b07b-9d3697a9ca48 None admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:42.997289 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f76d0f6-8773-485d-b07b-9d3697a9ca48 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:01:42.997591 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f76d0f6-8773-485d-b07b-9d3697a9ca48 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:42.999502 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f76d0f6-8773-485d-b07b-9d3697a9ca48 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:43.006535 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 589/1166] 10.222.0.49 () {56 vars in 1115 bytes} [Fri Sep 9 12:01:42 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:01:45.130490 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59a91d0-a84a-4115-ae90-bcc57b917d26 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:45.131052 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59a91d0-a84a-4115-ae90-bcc57b917d26 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:45.131052 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59a91d0-a84a-4115-ae90-bcc57b917d26 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:45.132408 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 578/1167] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:01:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:01:45.137748 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6054dd57-c383-4b9b-a924-9864eac01b06 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:45.138173 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6054dd57-c383-4b9b-a924-9864eac01b06 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:45.138285 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6054dd57-c383-4b9b-a924-9864eac01b06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:45.157388 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-6054dd57-c383-4b9b-a924-9864eac01b06 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:01:45.158573 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-6054dd57-c383-4b9b-a924-9864eac01b06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:01:45.231921 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 590/1168] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:01:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 12:01:45.500644 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59a91d0-a84a-4115-ae90-bcc57b917d26 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:45.599063 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59a91d0-a84a-4115-ae90-bcc57b917d26 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:45.632158 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af95711-7175-4f26-9da9-a4b1d5c12e3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:45.632763 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af95711-7175-4f26-9da9-a4b1d5c12e3a None admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:45.633030 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af95711-7175-4f26-9da9-a4b1d5c12e3a None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:45.633294 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af95711-7175-4f26-9da9-a4b1d5c12e3a None admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:45.633944 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af95711-7175-4f26-9da9-a4b1d5c12e3a None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:45.635320 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af95711-7175-4f26-9da9-a4b1d5c12e3a None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:45.651376 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 579/1169] 10.222.0.49 () {60 vars in 1138 bytes} [Fri Sep 9 12:01:45 2022] POST /identity/v3/domains => generated 267 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:01:47.807209 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93a565d-0218-4bc5-b939-abd264c65afa None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:47.807749 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93a565d-0218-4bc5-b939-abd264c65afa None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:47.807851 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93a565d-0218-4bc5-b939-abd264c65afa None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:47.809116 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 591/1170] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:01:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:01:47.815236 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c85a760-0e3e-4cb4-80e0-96b1c7d92688 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:47.815632 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c85a760-0e3e-4cb4-80e0-96b1c7d92688 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:47.815770 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c85a760-0e3e-4cb4-80e0-96b1c7d92688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:47.833987 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-9c85a760-0e3e-4cb4-80e0-96b1c7d92688 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:01:47.835178 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-9c85a760-0e3e-4cb4-80e0-96b1c7d92688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:01:47.863429 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:47.901883 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 580/1171] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:01:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 12:01:48.168059 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93a565d-0218-4bc5-b939-abd264c65afa None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:48.258403 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:48.265671 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93a565d-0218-4bc5-b939-abd264c65afa None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:48.298705 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d899fd6-14dd-4a1e-84b4-fd495b800306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:48.299347 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d899fd6-14dd-4a1e-84b4-fd495b800306 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:48.299621 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d899fd6-14dd-4a1e-84b4-fd495b800306 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:48.299885 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d899fd6-14dd-4a1e-84b4-fd495b800306 None admin] PATH_INFO: `/v3/domains/67d8bc5780cf43cdbd0d6929f8efc1cb` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:48.309693 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d899fd6-14dd-4a1e-84b4-fd495b800306 None admin] RBAC: Authorizing `identity:get_domain(domain_id=67d8bc5780cf43cdbd0d6929f8efc1cb)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:48.311770 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d899fd6-14dd-4a1e-84b4-fd495b800306 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:48.313902 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 592/1172] 10.222.0.49 () {56 vars in 1183 bytes} [Fri Sep 9 12:01:48 2022] GET /identity/v3/domains/67d8bc5780cf43cdbd0d6929f8efc1cb => generated 267 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:01:48.319742 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c85a760-0e3e-4cb4-80e0-96b1c7d92688 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:48.413526 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c85a760-0e3e-4cb4-80e0-96b1c7d92688 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:48.448098 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eae9e17-c523-4333-b878-1d62387338ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:48.448621 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eae9e17-c523-4333-b878-1d62387338ee None admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:48.448894 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eae9e17-c523-4333-b878-1d62387338ee None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:48.449159 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eae9e17-c523-4333-b878-1d62387338ee None admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:48.449800 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eae9e17-c523-4333-b878-1d62387338ee None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:48.451236 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eae9e17-c523-4333-b878-1d62387338ee None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:48.476871 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 581/1173] 10.222.0.49 () {60 vars in 1141 bytes} [Fri Sep 9 12:01:48 2022] POST /identity/v3/projects => generated 361 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:01:50.648787 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d83c9e-f801-4ee7-a07f-ff8e7a783b56 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:50.649196 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d83c9e-f801-4ee7-a07f-ff8e7a783b56 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:50.649332 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d83c9e-f801-4ee7-a07f-ff8e7a783b56 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:50.650666 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 593/1174] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:01:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:01:50.656408 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58763a66-3f65-4adc-808b-48ca16853fff None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:50.656800 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58763a66-3f65-4adc-808b-48ca16853fff None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:50.656935 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58763a66-3f65-4adc-808b-48ca16853fff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:50.674650 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-58763a66-3f65-4adc-808b-48ca16853fff None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:01:50.675856 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-58763a66-3f65-4adc-808b-48ca16853fff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:01:50.690191 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:50.741298 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 582/1175] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:01:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 12:01:51.010581 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d83c9e-f801-4ee7-a07f-ff8e7a783b56 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:51.110343 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d83c9e-f801-4ee7-a07f-ff8e7a783b56 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:51.142232 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a407e387-4fc4-4b5b-a6f5-e3e63b424c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:51.142852 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a407e387-4fc4-4b5b-a6f5-e3e63b424c12 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:51.143161 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a407e387-4fc4-4b5b-a6f5-e3e63b424c12 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:51.143434 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a407e387-4fc4-4b5b-a6f5-e3e63b424c12 None admin] PATH_INFO: `/v3/domains/67d8bc5780cf43cdbd0d6929f8efc1cb` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:51.144544 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a407e387-4fc4-4b5b-a6f5-e3e63b424c12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=67d8bc5780cf43cdbd0d6929f8efc1cb)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:51.145969 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a407e387-4fc4-4b5b-a6f5-e3e63b424c12 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:51.147557 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 594/1176] 10.222.0.49 () {56 vars in 1183 bytes} [Fri Sep 9 12:01:51 2022] GET /identity/v3/domains/67d8bc5780cf43cdbd0d6929f8efc1cb => generated 267 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:01:51.153184 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58763a66-3f65-4adc-808b-48ca16853fff None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:51.250544 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58763a66-3f65-4adc-808b-48ca16853fff None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:51.284374 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b4cfaf-e944-4fee-a026-870d514fff56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:51.284906 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b4cfaf-e944-4fee-a026-870d514fff56 None admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:51.285183 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b4cfaf-e944-4fee-a026-870d514fff56 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:51.285451 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b4cfaf-e944-4fee-a026-870d514fff56 None admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:51.286106 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b4cfaf-e944-4fee-a026-870d514fff56 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:51.287561 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b4cfaf-e944-4fee-a026-870d514fff56 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:51.308445 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 583/1177] 10.222.0.49 () {60 vars in 1132 bytes} [Fri Sep 9 12:01:51 2022] POST /identity/v3/users => generated 311 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:01:53.478443 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cd27f8-262f-4ce6-9e09-f85610decc28 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:53.479048 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cd27f8-262f-4ce6-9e09-f85610decc28 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:53.479048 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cd27f8-262f-4ce6-9e09-f85610decc28 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:53.480388 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 595/1178] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:01:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:01:53.485808 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfdb1ff-8131-4138-bcb8-a52ef07dad56 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:53.486229 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfdb1ff-8131-4138-bcb8-a52ef07dad56 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:53.486365 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfdb1ff-8131-4138-bcb8-a52ef07dad56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:53.499166 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:53.504936 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-acfdb1ff-8131-4138-bcb8-a52ef07dad56 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:01:53.506378 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-acfdb1ff-8131-4138-bcb8-a52ef07dad56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:01:53.572816 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 584/1179] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:01:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 12:01:53.841896 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cd27f8-262f-4ce6-9e09-f85610decc28 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:53.867794 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:53.940809 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cd27f8-262f-4ce6-9e09-f85610decc28 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:53.972601 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09702495-efba-428b-9757-7e3b0b481a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:53.973215 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09702495-efba-428b-9757-7e3b0b481a54 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:53.973491 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09702495-efba-428b-9757-7e3b0b481a54 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:53.973759 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09702495-efba-428b-9757-7e3b0b481a54 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:53.984415 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09702495-efba-428b-9757-7e3b0b481a54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:01:53.989138 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-09702495-efba-428b-9757-7e3b0b481a54 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 12:01:53.989138 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:01:53.989138 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:01:53.989138 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:01:53.989138 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:01:53.989138 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:01:53.989138 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:01:53.989138 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:01:53.989138 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:01:53.989138 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:01:53.989138 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:01:53.989138 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:01:53.989138 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:01:53.989138 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:01:53.989138 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:01:53.989138 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:01:53.990229 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:01:53.990229 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:01:53.990229 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:01:53.990229 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:01:53.990229 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:01:53.990229 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:01:53.990229 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:01:53.990229 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:01:53.990229 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:01:53.990229 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:01:53.990229 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:01:53.990229 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 12:01:53.990229 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:01:53.990229 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:01:53.990229 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:01:53.990229 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:01:53.991183 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:01:53.991183 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:01:53.991183 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:01:53.991183 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:01:53.991183 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:01:53.991183 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:01:53.991183 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:01:53.991183 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:01:53.991183 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:01:53.991183 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:01:53.991183 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 12:01:53.991183 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:01:53.991183 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 596/1180] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Sep 9 12:01:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:01:53.996146 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfdb1ff-8131-4138-bcb8-a52ef07dad56 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:54.092649 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfdb1ff-8131-4138-bcb8-a52ef07dad56 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:54.125891 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca44893-e33d-4d0c-ad60-92deb0cb4163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:54.126484 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca44893-e33d-4d0c-ad60-92deb0cb4163 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:54.126758 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca44893-e33d-4d0c-ad60-92deb0cb4163 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:54.127053 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca44893-e33d-4d0c-ad60-92deb0cb4163 None admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:54.127696 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca44893-e33d-4d0c-ad60-92deb0cb4163 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:01:54.127993 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca44893-e33d-4d0c-ad60-92deb0cb4163 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:54.129344 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca44893-e33d-4d0c-ad60-92deb0cb4163 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:54.131477 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:54.138058 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 585/1181] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 9 12:01:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:01:54.144158 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09702495-efba-428b-9757-7e3b0b481a54 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:54.226446 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:54.241875 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09702495-efba-428b-9757-7e3b0b481a54 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:54.274169 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf79db1-ed82-4ea9-acbb-7f500101a1c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:54.274787 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf79db1-ed82-4ea9-acbb-7f500101a1c4 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:54.275092 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf79db1-ed82-4ea9-acbb-7f500101a1c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:54.275361 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf79db1-ed82-4ea9-acbb-7f500101a1c4 None admin] PATH_INFO: `/v3/users/e435d847b2fb409da3e18a911e103d0f` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:54.288690 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf79db1-ed82-4ea9-acbb-7f500101a1c4 None admin] RBAC: Authorizing `identity:get_user(user_id=e435d847b2fb409da3e18a911e103d0f)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:54.290106 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf79db1-ed82-4ea9-acbb-7f500101a1c4 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:54.291689 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 597/1182] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Sep 9 12:01:54 2022] GET /identity/v3/users/e435d847b2fb409da3e18a911e103d0f => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:01:54.297532 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca44893-e33d-4d0c-ad60-92deb0cb4163 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:54.395874 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca44893-e33d-4d0c-ad60-92deb0cb4163 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:54.428375 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade2afdc-e695-476b-97af-4efa11f42318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:54.428966 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade2afdc-e695-476b-97af-4efa11f42318 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:54.429243 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade2afdc-e695-476b-97af-4efa11f42318 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:54.429513 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade2afdc-e695-476b-97af-4efa11f42318 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:54.491409 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:54.495493 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade2afdc-e695-476b-97af-4efa11f42318 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:54.496930 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade2afdc-e695-476b-97af-4efa11f42318 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:54.593175 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 586/1183] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 9 12:01:54 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 12:01:54.599134 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf79db1-ed82-4ea9-acbb-7f500101a1c4 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:54.695945 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf79db1-ed82-4ea9-acbb-7f500101a1c4 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:54.729625 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1978eff1-64b7-410c-b02b-bca9a13c1c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:54.730230 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1978eff1-64b7-410c-b02b-bca9a13c1c1b None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:54.730501 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1978eff1-64b7-410c-b02b-bca9a13c1c1b None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:54.730767 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1978eff1-64b7-410c-b02b-bca9a13c1c1b None admin] PATH_INFO: `/v3/projects/36455ceb195e4229b0d58ab9a5f56f99` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:54.731884 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1978eff1-64b7-410c-b02b-bca9a13c1c1b None admin] RBAC: Authorizing `identity:get_project(project_id=36455ceb195e4229b0d58ab9a5f56f99)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:54.733733 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1978eff1-64b7-410c-b02b-bca9a13c1c1b None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:54.735351 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 598/1184] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Sep 9 12:01:54 2022] GET /identity/v3/projects/36455ceb195e4229b0d58ab9a5f56f99 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:01:54.741572 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade2afdc-e695-476b-97af-4efa11f42318 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:54.835411 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:54.839465 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade2afdc-e695-476b-97af-4efa11f42318 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:54.872700 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b4840b-b643-46c7-8de9-06ab743766c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:54.873913 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b4840b-b643-46c7-8de9-06ab743766c8 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:54.874247 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b4840b-b643-46c7-8de9-06ab743766c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:54.874518 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b4840b-b643-46c7-8de9-06ab743766c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:54.875257 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b4840b-b643-46c7-8de9-06ab743766c8 None admin] RBAC: Adding query filter params (role.id=3f2ca33902a04b6dbc2ef7d06d7289d3, scope.project.id=36455ceb195e4229b0d58ab9a5f56f99, user.id=e435d847b2fb409da3e18a911e103d0f) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:01:54.875560 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b4840b-b643-46c7-8de9-06ab743766c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:54.876978 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b4840b-b643-46c7-8de9-06ab743766c8 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:54.882135 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 587/1185] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Sep 9 12:01:54 2022] GET /identity/v3/role_assignments?user.id=e435d847b2fb409da3e18a911e103d0f&scope.project.id=36455ceb195e4229b0d58ab9a5f56f99&role.id=3f2ca33902a04b6dbc2ef7d06d7289d3 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:01:57.017128 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6defec-e6c2-43af-8edb-91188fbf0828 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:57.017657 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6defec-e6c2-43af-8edb-91188fbf0828 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:57.017657 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6defec-e6c2-43af-8edb-91188fbf0828 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:57.019039 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 599/1186] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:01:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:01:57.024175 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95d6fd4-8c47-435b-b084-0c56b9364d5c None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:57.024618 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95d6fd4-8c47-435b-b084-0c56b9364d5c None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:57.024710 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95d6fd4-8c47-435b-b084-0c56b9364d5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:57.042432 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-e95d6fd4-8c47-435b-b084-0c56b9364d5c None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:01:57.043635 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-e95d6fd4-8c47-435b-b084-0c56b9364d5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:01:57.110016 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 588/1187] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:01:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 12:01:57.375989 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6defec-e6c2-43af-8edb-91188fbf0828 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:57.387961 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:57.473345 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6defec-e6c2-43af-8edb-91188fbf0828 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:57.506011 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77d06f6-0086-415e-ac3a-2d898c65b2ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:57.506628 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77d06f6-0086-415e-ac3a-2d898c65b2ce None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:57.506937 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77d06f6-0086-415e-ac3a-2d898c65b2ce None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:57.507200 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77d06f6-0086-415e-ac3a-2d898c65b2ce None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:57.517039 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77d06f6-0086-415e-ac3a-2d898c65b2ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:01:57.521767 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-d77d06f6-0086-415e-ac3a-2d898c65b2ce None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 12:01:57.521767 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:01:57.521767 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:01:57.521767 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:01:57.521767 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:01:57.521767 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:01:57.521767 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:01:57.521767 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:01:57.521767 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:01:57.521767 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:01:57.521767 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:01:57.521767 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:01:57.521767 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:01:57.521767 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:01:57.521767 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:01:57.521767 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:01:57.522944 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:01:57.522944 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:01:57.522944 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:01:57.522944 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:01:57.522944 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:01:57.522944 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:01:57.522944 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:01:57.522944 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:01:57.522944 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:01:57.522944 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:01:57.522944 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:01:57.522944 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 12:01:57.522944 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:01:57.522944 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:01:57.522944 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:01:57.522944 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:01:57.523807 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:01:57.523807 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:01:57.523807 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:01:57.523807 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:01:57.523807 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:01:57.523807 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:01:57.523807 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:01:57.523807 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:01:57.523807 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:01:57.523807 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:01:57.523807 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 12:01:57.523807 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:01:57.523807 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 600/1188] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Sep 9 12:01:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:01:57.529127 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95d6fd4-8c47-435b-b084-0c56b9364d5c None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:57.617605 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:57.626635 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95d6fd4-8c47-435b-b084-0c56b9364d5c None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:57.660358 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695441bd-ba94-41ae-ad92-4a34e0afbf0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:57.660966 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695441bd-ba94-41ae-ad92-4a34e0afbf0d None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:57.661262 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695441bd-ba94-41ae-ad92-4a34e0afbf0d None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:57.661516 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695441bd-ba94-41ae-ad92-4a34e0afbf0d None admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:57.662169 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695441bd-ba94-41ae-ad92-4a34e0afbf0d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:01:57.662470 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695441bd-ba94-41ae-ad92-4a34e0afbf0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:57.663867 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695441bd-ba94-41ae-ad92-4a34e0afbf0d None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:57.672392 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 589/1189] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 9 12:01:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:01:57.678236 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77d06f6-0086-415e-ac3a-2d898c65b2ce None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:57.744988 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:57.773828 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77d06f6-0086-415e-ac3a-2d898c65b2ce None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:57.806304 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f044d038-e0a9-4721-b1d4-603a6b7e78a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:57.806942 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f044d038-e0a9-4721-b1d4-603a6b7e78a6 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:57.807216 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f044d038-e0a9-4721-b1d4-603a6b7e78a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:57.807480 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f044d038-e0a9-4721-b1d4-603a6b7e78a6 None admin] PATH_INFO: `/v3/users/e435d847b2fb409da3e18a911e103d0f` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:57.808573 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f044d038-e0a9-4721-b1d4-603a6b7e78a6 None admin] RBAC: Authorizing `identity:get_user(user_id=e435d847b2fb409da3e18a911e103d0f)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:57.809980 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f044d038-e0a9-4721-b1d4-603a6b7e78a6 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:57.812180 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 601/1190] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Sep 9 12:01:57 2022] GET /identity/v3/users/e435d847b2fb409da3e18a911e103d0f => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:01:57.818148 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695441bd-ba94-41ae-ad92-4a34e0afbf0d None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:57.918519 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695441bd-ba94-41ae-ad92-4a34e0afbf0d None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:57.952032 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c633e02-20d5-4781-b872-90884fbb3694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:57.952650 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c633e02-20d5-4781-b872-90884fbb3694 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:57.952926 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c633e02-20d5-4781-b872-90884fbb3694 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:57.953193 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c633e02-20d5-4781-b872-90884fbb3694 None admin] PATH_INFO: `/v3/projects/36455ceb195e4229b0d58ab9a5f56f99` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:57.954300 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c633e02-20d5-4781-b872-90884fbb3694 None admin] RBAC: Authorizing `identity:get_project(project_id=36455ceb195e4229b0d58ab9a5f56f99)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:57.955748 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c633e02-20d5-4781-b872-90884fbb3694 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:57.957356 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 590/1191] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Sep 9 12:01:57 2022] GET /identity/v3/projects/36455ceb195e4229b0d58ab9a5f56f99 => generated 361 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:01:57.963297 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f044d038-e0a9-4721-b1d4-603a6b7e78a6 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:01:58.057175 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f044d038-e0a9-4721-b1d4-603a6b7e78a6 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:01:58.089743 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b29481e-ffe5-4dc9-a3b2-3e090fee0ac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:01:58.090376 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b29481e-ffe5-4dc9-a3b2-3e090fee0ac0 None admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:01:58.090650 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b29481e-ffe5-4dc9-a3b2-3e090fee0ac0 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:01:58.090937 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b29481e-ffe5-4dc9-a3b2-3e090fee0ac0 None admin] PATH_INFO: `/v3/projects/36455ceb195e4229b0d58ab9a5f56f99/users/e435d847b2fb409da3e18a911e103d0f/roles/3f2ca33902a04b6dbc2ef7d06d7289d3` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:01:58.092918 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b29481e-ffe5-4dc9-a3b2-3e090fee0ac0 None admin] RBAC: Authorizing `identity:create_grant(project_id=36455ceb195e4229b0d58ab9a5f56f99, user_id=e435d847b2fb409da3e18a911e103d0f, role_id=3f2ca33902a04b6dbc2ef7d06d7289d3)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:01:58.094320 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b29481e-ffe5-4dc9-a3b2-3e090fee0ac0 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:01:58.101778 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:01:58.108386 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 602/1192] 10.222.0.49 () {58 vars in 1439 bytes} [Fri Sep 9 12:01:57 2022] PUT /identity/v3/projects/36455ceb195e4229b0d58ab9a5f56f99/users/e435d847b2fb409da3e18a911e103d0f/roles/3f2ca33902a04b6dbc2ef7d06d7289d3 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:02:00.209882 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ac8f53-be28-40da-bfd7-fa3395fec5e2 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:02:00.210411 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ac8f53-be28-40da-bfd7-fa3395fec5e2 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:02:00.210411 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ac8f53-be28-40da-bfd7-fa3395fec5e2 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:02:00.211843 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 591/1193] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:02:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:02:00.217053 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3594f7c1-991d-4ad3-8db7-61d4eae64dff None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:02:00.217502 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3594f7c1-991d-4ad3-8db7-61d4eae64dff None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:02:00.217768 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3594f7c1-991d-4ad3-8db7-61d4eae64dff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:02:00.235871 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-3594f7c1-991d-4ad3-8db7-61d4eae64dff None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:02:00.237033 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-3594f7c1-991d-4ad3-8db7-61d4eae64dff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:02:00.300044 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 603/1194] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:02:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 12:02:00.571519 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ac8f53-be28-40da-bfd7-fa3395fec5e2 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:02:00.596483 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:02:00.672785 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ac8f53-be28-40da-bfd7-fa3395fec5e2 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:02:00.704826 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f9bcea-1f72-4d68-95a1-55fbe9a3fda0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:02:00.705441 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f9bcea-1f72-4d68-95a1-55fbe9a3fda0 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:02:00.705715 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f9bcea-1f72-4d68-95a1-55fbe9a3fda0 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:02:00.705981 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f9bcea-1f72-4d68-95a1-55fbe9a3fda0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:02:00.717433 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f9bcea-1f72-4d68-95a1-55fbe9a3fda0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:02:00.722156 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-64f9bcea-1f72-4d68-95a1-55fbe9a3fda0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 12:02:00.722156 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:02:00.722156 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:02:00.722156 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:02:00.722156 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:02:00.722156 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:02:00.722156 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:02:00.722156 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:02:00.722156 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:02:00.722156 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:02:00.722156 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:02:00.722156 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:02:00.722156 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:02:00.722156 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:02:00.722156 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:02:00.722156 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:02:00.723325 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:02:00.723325 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:02:00.723325 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:02:00.723325 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:02:00.723325 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:02:00.723325 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:02:00.723325 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:02:00.723325 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:02:00.723325 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:02:00.723325 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:02:00.723325 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:02:00.723325 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 12:02:00.723325 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:02:00.723325 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:02:00.723325 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:02:00.723325 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:02:00.724405 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:02:00.724405 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:02:00.724405 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:02:00.724405 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:02:00.724405 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:02:00.724405 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:02:00.724405 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:02:00.724405 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:02:00.724405 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:02:00.724405 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:02:00.724405 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 12:02:00.724405 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:02:00.724405 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 592/1195] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Sep 9 12:02:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:02:00.729428 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3594f7c1-991d-4ad3-8db7-61d4eae64dff None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:02:00.823160 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3594f7c1-991d-4ad3-8db7-61d4eae64dff None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:02:00.848625 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:02:00.856261 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d774846a-4df1-496b-bb93-0d4fba7d3ed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:02:00.856856 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d774846a-4df1-496b-bb93-0d4fba7d3ed5 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:02:00.857647 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d774846a-4df1-496b-bb93-0d4fba7d3ed5 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:02:00.857938 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d774846a-4df1-496b-bb93-0d4fba7d3ed5 None admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:02:00.858584 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d774846a-4df1-496b-bb93-0d4fba7d3ed5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:02:00.858879 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d774846a-4df1-496b-bb93-0d4fba7d3ed5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:02:00.860259 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d774846a-4df1-496b-bb93-0d4fba7d3ed5 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:02:00.868008 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 604/1196] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 9 12:02:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:02:00.874434 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f9bcea-1f72-4d68-95a1-55fbe9a3fda0 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:02:00.953033 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:02:00.970778 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f9bcea-1f72-4d68-95a1-55fbe9a3fda0 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:02:01.003791 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0dd912-ba11-46e3-a41f-9484657c6e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:02:01.004402 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0dd912-ba11-46e3-a41f-9484657c6e38 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:02:01.004676 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0dd912-ba11-46e3-a41f-9484657c6e38 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:02:01.004944 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0dd912-ba11-46e3-a41f-9484657c6e38 None admin] PATH_INFO: `/v3/users/e435d847b2fb409da3e18a911e103d0f` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:02:01.006046 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0dd912-ba11-46e3-a41f-9484657c6e38 None admin] RBAC: Authorizing `identity:get_user(user_id=e435d847b2fb409da3e18a911e103d0f)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:02:01.007492 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0dd912-ba11-46e3-a41f-9484657c6e38 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:02:01.009035 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 593/1197] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Sep 9 12:02:00 2022] GET /identity/v3/users/e435d847b2fb409da3e18a911e103d0f => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:02:01.014753 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d774846a-4df1-496b-bb93-0d4fba7d3ed5 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:02:01.111039 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d774846a-4df1-496b-bb93-0d4fba7d3ed5 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:02:01.143047 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a890050f-b7f9-420e-8b9a-6e72a5bbb302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:02:01.143640 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a890050f-b7f9-420e-8b9a-6e72a5bbb302 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:02:01.143925 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a890050f-b7f9-420e-8b9a-6e72a5bbb302 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:02:01.144196 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a890050f-b7f9-420e-8b9a-6e72a5bbb302 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:02:01.198839 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:02:01.209348 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a890050f-b7f9-420e-8b9a-6e72a5bbb302 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:02:01.210780 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a890050f-b7f9-420e-8b9a-6e72a5bbb302 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:02:01.305343 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 605/1198] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 9 12:02:01 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 12:02:01.312580 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0dd912-ba11-46e3-a41f-9484657c6e38 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:02:01.415920 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0dd912-ba11-46e3-a41f-9484657c6e38 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:02:01.450825 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7443b99-91fb-40aa-9990-51bfb3e91c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:02:01.452298 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7443b99-91fb-40aa-9990-51bfb3e91c5b None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:02:01.452588 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7443b99-91fb-40aa-9990-51bfb3e91c5b None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:02:01.452859 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7443b99-91fb-40aa-9990-51bfb3e91c5b None admin] PATH_INFO: `/v3/projects/36455ceb195e4229b0d58ab9a5f56f99` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:02:01.453958 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7443b99-91fb-40aa-9990-51bfb3e91c5b None admin] RBAC: Authorizing `identity:get_project(project_id=36455ceb195e4229b0d58ab9a5f56f99)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:02:01.455415 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7443b99-91fb-40aa-9990-51bfb3e91c5b None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:02:01.457046 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 594/1199] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Sep 9 12:02:01 2022] GET /identity/v3/projects/36455ceb195e4229b0d58ab9a5f56f99 => generated 361 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:02:01.463206 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a890050f-b7f9-420e-8b9a-6e72a5bbb302 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:02:01.550099 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:02:01.559718 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a890050f-b7f9-420e-8b9a-6e72a5bbb302 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:02:01.593376 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032b5926-1cf8-4725-9387-1e7eff079e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:02:01.593970 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032b5926-1cf8-4725-9387-1e7eff079e81 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:02:01.594244 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032b5926-1cf8-4725-9387-1e7eff079e81 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:02:01.594509 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032b5926-1cf8-4725-9387-1e7eff079e81 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:02:01.595246 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032b5926-1cf8-4725-9387-1e7eff079e81 None admin] RBAC: Adding query filter params (role.id=3f2ca33902a04b6dbc2ef7d06d7289d3, scope.project.id=36455ceb195e4229b0d58ab9a5f56f99, user.id=e435d847b2fb409da3e18a911e103d0f) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:02:01.595544 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032b5926-1cf8-4725-9387-1e7eff079e81 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:02:01.596945 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032b5926-1cf8-4725-9387-1e7eff079e81 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:02:01.602394 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 606/1200] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Sep 9 12:02:01 2022] GET /identity/v3/role_assignments?user.id=e435d847b2fb409da3e18a911e103d0f&scope.project.id=36455ceb195e4229b0d58ab9a5f56f99&role.id=3f2ca33902a04b6dbc2ef7d06d7289d3 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:02:11.446784 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=91 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:02:11.599932 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=23 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:02:57.450981 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f210b7-607a-42ab-b6a6-5b798b538b3e None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:02:57.450981 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f210b7-607a-42ab-b6a6-5b798b538b3e None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:02:57.450981 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f210b7-607a-42ab-b6a6-5b798b538b3e None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:02:57.452414 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 595/1201] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:02:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 12:02:57.457996 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cc6930-fcaa-496e-bfef-edb8ada2345a None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:02:57.458599 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cc6930-fcaa-496e-bfef-edb8ada2345a None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:02:57.458599 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cc6930-fcaa-496e-bfef-edb8ada2345a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:02:57.478579 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-43cc6930-fcaa-496e-bfef-edb8ada2345a None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:02:57.480712 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-43cc6930-fcaa-496e-bfef-edb8ada2345a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:02:57.579363 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 607/1202] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:02:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 12:02:57.855337 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f210b7-607a-42ab-b6a6-5b798b538b3e None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:02:57.986836 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f210b7-607a-42ab-b6a6-5b798b538b3e None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:02:58.032750 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d337eee-f9eb-4722-8b0d-44cece27a54a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:02:58.033416 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d337eee-f9eb-4722-8b0d-44cece27a54a None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:02:58.033698 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d337eee-f9eb-4722-8b0d-44cece27a54a None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:02:58.033970 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d337eee-f9eb-4722-8b0d-44cece27a54a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:02:58.045556 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d337eee-f9eb-4722-8b0d-44cece27a54a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:02:58.048781 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d337eee-f9eb-4722-8b0d-44cece27a54a None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:02:58.062828 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-3d337eee-f9eb-4722-8b0d-44cece27a54a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:02:58.062828 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:02:58.062828 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:02:58.062828 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:02:58.062828 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:02:58.062828 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:02:58.062828 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:02:58.062828 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:02:58.062828 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:02:58.062828 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:02:58.062828 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:02:58.062828 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:02:58.062828 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:02:58.062828 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:02:58.062828 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:02:58.062828 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:02:58.064162 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:02:58.064162 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:02:58.064162 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:02:58.064162 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:02:58.064162 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:02:58.064162 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:02:58.064162 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:02:58.064162 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 12:02:58.064162 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:02:58.064162 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:02:58.064162 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:02:58.064162 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:02:58.064162 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:02:58.064162 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:02:58.064162 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:02:58.064162 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:02:58.064162 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:02:58.065203 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:02:58.065203 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:02:58.065203 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:02:58.065203 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 596/1203] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 12:02:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:02:58.070352 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cc6930-fcaa-496e-bfef-edb8ada2345a None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:02:58.203326 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cc6930-fcaa-496e-bfef-edb8ada2345a None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:02:58.237611 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:02:58.247927 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf14ae8-846c-4e70-abdf-fe1279160e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:02:58.248691 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf14ae8-846c-4e70-abdf-fe1279160e8b None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:02:58.249066 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf14ae8-846c-4e70-abdf-fe1279160e8b None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:02:58.249403 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf14ae8-846c-4e70-abdf-fe1279160e8b None admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:02:58.250144 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf14ae8-846c-4e70-abdf-fe1279160e8b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:02:58.250510 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf14ae8-846c-4e70-abdf-fe1279160e8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:02:58.252020 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf14ae8-846c-4e70-abdf-fe1279160e8b None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:02:58.270360 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 608/1204] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 9 12:02:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:02:58.271801 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d337eee-f9eb-4722-8b0d-44cece27a54a None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:02:58.343588 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:02:58.420150 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d337eee-f9eb-4722-8b0d-44cece27a54a None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:02:58.459799 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8aadd58-f1fc-4d25-80e3-1c5230b9535c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:02:58.460411 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aadd58-f1fc-4d25-80e3-1c5230b9535c None admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:02:58.461369 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aadd58-f1fc-4d25-80e3-1c5230b9535c None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:02:58.461655 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aadd58-f1fc-4d25-80e3-1c5230b9535c None admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:02:58.462415 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8aadd58-f1fc-4d25-80e3-1c5230b9535c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:02:58.471019 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8aadd58-f1fc-4d25-80e3-1c5230b9535c None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:02:58.497994 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 597/1205] 10.222.0.49 () {60 vars in 1132 bytes} [Fri Sep 9 12:02:58 2022] POST /identity/v3/users => generated 251 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:03:01.035186 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eb30b2-f327-40f6-a2e5-ab34f399efa3 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:03:01.035742 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eb30b2-f327-40f6-a2e5-ab34f399efa3 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:03:01.035988 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eb30b2-f327-40f6-a2e5-ab34f399efa3 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:03:01.037816 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 609/1206] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:03:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:03:01.044973 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e52288-7c25-437f-a4bb-226b9f9eac5f None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:03:01.045473 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e52288-7c25-437f-a4bb-226b9f9eac5f None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:03:01.045574 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e52288-7c25-437f-a4bb-226b9f9eac5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:03:01.068086 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-f0e52288-7c25-437f-a4bb-226b9f9eac5f None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:03:01.069424 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-f0e52288-7c25-437f-a4bb-226b9f9eac5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:03:01.158260 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 598/1207] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:03:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 12:03:01.457997 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95eb30b2-f327-40f6-a2e5-ab34f399efa3 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:03:01.585552 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95eb30b2-f327-40f6-a2e5-ab34f399efa3 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:03:01.633039 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4916529a-2337-4c10-892d-85c708cab370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:03:01.633719 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4916529a-2337-4c10-892d-85c708cab370 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:03:01.634007 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4916529a-2337-4c10-892d-85c708cab370 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:03:01.635022 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4916529a-2337-4c10-892d-85c708cab370 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:03:01.651378 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4916529a-2337-4c10-892d-85c708cab370 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:03:01.656565 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-4916529a-2337-4c10-892d-85c708cab370 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 12:03:01.656565 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:03:01.656565 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:03:01.656565 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:03:01.656565 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:03:01.656565 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:03:01.656565 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:03:01.656565 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:03:01.656565 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:03:01.656565 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:03:01.656565 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:03:01.656565 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:03:01.656565 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:03:01.656565 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:03:01.656565 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:03:01.656565 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:03:01.659328 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:03:01.659328 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:03:01.659328 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:03:01.659328 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:03:01.659328 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:03:01.659328 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:03:01.659328 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:03:01.659328 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:03:01.659328 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:03:01.659328 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:03:01.659328 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:03:01.659328 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 12:03:01.659328 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:03:01.659328 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:03:01.659328 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:03:01.659328 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:03:01.660384 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:03:01.660384 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:03:01.660384 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:03:01.660384 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:03:01.660384 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:03:01.660384 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:03:01.660384 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:03:01.660384 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:03:01.660384 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:03:01.660384 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:03:01.660384 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 12:03:01.660384 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:03:01.660384 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 610/1208] 10.222.0.49 () {56 vars in 1102 bytes} [Fri Sep 9 12:03:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:03:01.664172 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e52288-7c25-437f-a4bb-226b9f9eac5f None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:03:01.710430 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:03:01.779876 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e52288-7c25-437f-a4bb-226b9f9eac5f None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:03:01.818973 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9490b73-4f8b-407d-bee8-9dc304d4cdbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:03:01.819583 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9490b73-4f8b-407d-bee8-9dc304d4cdbf None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:03:01.819862 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9490b73-4f8b-407d-bee8-9dc304d4cdbf None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:03:01.820126 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9490b73-4f8b-407d-bee8-9dc304d4cdbf None admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:03:01.820803 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9490b73-4f8b-407d-bee8-9dc304d4cdbf None admin] RBAC: Adding query filter params (name=service) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:03:01.821093 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9490b73-4f8b-407d-bee8-9dc304d4cdbf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:03:01.823420 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9490b73-4f8b-407d-bee8-9dc304d4cdbf None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:03:01.834971 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 599/1209] 10.222.0.49 () {56 vars in 1103 bytes} [Fri Sep 9 12:03:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:03:01.838322 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4916529a-2337-4c10-892d-85c708cab370 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:03:01.908084 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:03:01.969662 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4916529a-2337-4c10-892d-85c708cab370 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:03:02.009096 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78abb049-8547-412e-83cb-75b27929c63e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:03:02.010458 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78abb049-8547-412e-83cb-75b27929c63e None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:03:02.010737 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78abb049-8547-412e-83cb-75b27929c63e None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:03:02.011063 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78abb049-8547-412e-83cb-75b27929c63e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:03:02.025522 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78abb049-8547-412e-83cb-75b27929c63e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:03:02.027434 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78abb049-8547-412e-83cb-75b27929c63e None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:03:02.042542 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-78abb049-8547-412e-83cb-75b27929c63e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:03:02.042542 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:03:02.042542 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:03:02.042542 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:03:02.042542 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:03:02.042542 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:03:02.042542 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:03:02.042542 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:03:02.042542 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:03:02.042542 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:03:02.042542 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:03:02.042542 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:03:02.042542 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:03:02.042542 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:03:02.042542 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:03:02.042542 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:03:02.044255 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:03:02.044255 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:03:02.044255 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:03:02.044255 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:03:02.044255 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:03:02.044255 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:03:02.044255 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:03:02.044255 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 12:03:02.044255 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:03:02.044255 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:03:02.044255 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:03:02.044255 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:03:02.044255 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:03:02.044255 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:03:02.044255 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:03:02.044255 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:03:02.044255 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:03:02.045495 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:03:02.045495 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:03:02.045495 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:03:02.045495 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 611/1210] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 12:03:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:03:02.049571 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9490b73-4f8b-407d-bee8-9dc304d4cdbf None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:03:02.188063 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:03:02.192009 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9490b73-4f8b-407d-bee8-9dc304d4cdbf None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:03:02.237210 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e21828-05bb-457d-80f7-3d0e993eaef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:03:02.237819 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e21828-05bb-457d-80f7-3d0e993eaef6 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:03:02.238156 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e21828-05bb-457d-80f7-3d0e993eaef6 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:03:02.238486 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e21828-05bb-457d-80f7-3d0e993eaef6 None admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:03:02.239227 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e21828-05bb-457d-80f7-3d0e993eaef6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:03:02.239590 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e21828-05bb-457d-80f7-3d0e993eaef6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:03:02.241125 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e21828-05bb-457d-80f7-3d0e993eaef6 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:03:02.260774 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 600/1211] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 9 12:03:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:03:02.261591 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78abb049-8547-412e-83cb-75b27929c63e None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:03:02.373241 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78abb049-8547-412e-83cb-75b27929c63e None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:03:02.375069 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:03:02.416773 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f284aba-b32f-43fb-b84f-7db002390540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:03:02.417677 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f284aba-b32f-43fb-b84f-7db002390540 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:03:02.418119 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f284aba-b32f-43fb-b84f-7db002390540 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:03:02.418555 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f284aba-b32f-43fb-b84f-7db002390540 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:03:02.429105 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f284aba-b32f-43fb-b84f-7db002390540 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:03:02.430720 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f284aba-b32f-43fb-b84f-7db002390540 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:03:02.443639 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-3f284aba-b32f-43fb-b84f-7db002390540 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 09 12:03:02.443639 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:03:02.443639 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:03:02.443639 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:03:02.443639 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:03:02.443639 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:03:02.443639 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:03:02.443639 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:03:02.443639 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:03:02.443639 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:03:02.443639 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:03:02.443639 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:03:02.443639 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 12:03:02.443639 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 12:03:02.443639 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 12:03:02.443639 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 12:03:02.444719 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:03:02.444719 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:03:02.444719 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 12:03:02.444719 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:03:02.444719 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 12:03:02.444719 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:03:02.444719 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:03:02.444719 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:03:02.444719 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:03:02.444719 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:03:02.444719 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:03:02.444719 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 12:03:02.444719 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:03:02.444719 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:03:02.444719 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:03:02.444719 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:03:02.444719 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:03:02.444719 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:03:02.445680 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:03:02.445680 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:03:02.445680 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 12:03:02.445680 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 12:03:02.445680 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 12:03:02.445680 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 12:03:02.445680 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 12:03:02.445680 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 12:03:02.445680 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 09 12:03:02.445680 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:03:02.445680 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 612/1212] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 12:03:02 2022] GET /identity/v3/users/placement => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 12:03:02.451453 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e21828-05bb-457d-80f7-3d0e993eaef6 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:03:02.565866 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e21828-05bb-457d-80f7-3d0e993eaef6 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:03:02.607084 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697f3991-2c09-40dc-bca1-ad2488c438ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:03:02.607687 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697f3991-2c09-40dc-bca1-ad2488c438ce None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:03:02.607960 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697f3991-2c09-40dc-bca1-ad2488c438ce None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:03:02.608229 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697f3991-2c09-40dc-bca1-ad2488c438ce None admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:03:02.608946 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697f3991-2c09-40dc-bca1-ad2488c438ce None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:03:02.609248 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697f3991-2c09-40dc-bca1-ad2488c438ce None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:03:02.610663 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697f3991-2c09-40dc-bca1-ad2488c438ce None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:03:02.633995 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 601/1213] 10.222.0.49 () {56 vars in 1143 bytes} [Fri Sep 9 12:03:02 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:03:02.643062 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f284aba-b32f-43fb-b84f-7db002390540 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:03:02.749024 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f284aba-b32f-43fb-b84f-7db002390540 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:03:02.785443 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7646b5d8-d584-447c-a448-ff09b7d04f20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:03:02.786052 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7646b5d8-d584-447c-a448-ff09b7d04f20 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:03:02.786378 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7646b5d8-d584-447c-a448-ff09b7d04f20 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:03:02.786649 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7646b5d8-d584-447c-a448-ff09b7d04f20 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:03:02.826517 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:03:02.871191 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7646b5d8-d584-447c-a448-ff09b7d04f20 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:03:02.872818 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7646b5d8-d584-447c-a448-ff09b7d04f20 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:03:02.990084 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 613/1214] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 9 12:03:02 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 12:03:02.991322 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697f3991-2c09-40dc-bca1-ad2488c438ce None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:03:03.012119 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:03:03.115942 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697f3991-2c09-40dc-bca1-ad2488c438ce None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:03:03.161908 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f65b591-2dba-43b8-bf33-63f461e32f58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:03:03.162542 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f65b591-2dba-43b8-bf33-63f461e32f58 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:03:03.162817 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f65b591-2dba-43b8-bf33-63f461e32f58 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:03:03.163111 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f65b591-2dba-43b8-bf33-63f461e32f58 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:03:03.177724 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f65b591-2dba-43b8-bf33-63f461e32f58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:03:03.179222 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f65b591-2dba-43b8-bf33-63f461e32f58 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:03:03.199088 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-8f65b591-2dba-43b8-bf33-63f461e32f58 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:03:03.199088 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:03:03.199088 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:03:03.199088 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:03:03.199088 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:03:03.199088 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:03:03.199088 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:03:03.199088 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:03:03.199088 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:03:03.199088 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:03:03.199088 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:03:03.199088 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:03:03.199088 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:03:03.199088 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:03:03.199088 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:03:03.199088 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:03:03.200510 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:03:03.200510 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:03:03.200510 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:03:03.200510 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:03:03.200510 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:03:03.200510 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:03:03.200510 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:03:03.200510 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 12:03:03.200510 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:03:03.200510 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:03:03.200510 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:03:03.200510 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:03:03.200510 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:03:03.200510 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:03:03.200510 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:03:03.200510 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:03:03.200510 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:03:03.201882 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:03:03.201882 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:03:03.201882 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:03:03.201882 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 602/1215] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 12:03:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 12:03:03.206116 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7646b5d8-d584-447c-a448-ff09b7d04f20 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:03:03.277940 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:03:03.322479 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7646b5d8-d584-447c-a448-ff09b7d04f20 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:03:03.363095 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae70095c-b058-4667-ac09-cecff57c6e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:03:03.363763 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae70095c-b058-4667-ac09-cecff57c6e2b None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:03:03.364038 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae70095c-b058-4667-ac09-cecff57c6e2b None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:03:03.364304 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae70095c-b058-4667-ac09-cecff57c6e2b None admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:03:03.365610 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae70095c-b058-4667-ac09-cecff57c6e2b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:03:03.366100 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae70095c-b058-4667-ac09-cecff57c6e2b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:03:03.367477 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae70095c-b058-4667-ac09-cecff57c6e2b None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:03:03.379782 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 614/1216] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 9 12:03:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:03:03.386934 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f65b591-2dba-43b8-bf33-63f461e32f58 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:03:03.467983 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:03:03.502455 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f65b591-2dba-43b8-bf33-63f461e32f58 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:03:03.538927 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baac8a4d-6402-4918-9da7-858942cde286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:03:03.539656 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baac8a4d-6402-4918-9da7-858942cde286 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:03:03.539929 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baac8a4d-6402-4918-9da7-858942cde286 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:03:03.540199 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baac8a4d-6402-4918-9da7-858942cde286 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:03:03.548260 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baac8a4d-6402-4918-9da7-858942cde286 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:03:03.549812 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baac8a4d-6402-4918-9da7-858942cde286 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:03:03.562111 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-baac8a4d-6402-4918-9da7-858942cde286 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:03:03.562111 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:03:03.562111 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:03:03.562111 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:03:03.562111 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:03:03.562111 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:03:03.562111 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:03:03.562111 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:03:03.562111 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:03:03.562111 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:03:03.562111 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:03:03.562111 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:03:03.562111 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 12:03:03.562111 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 12:03:03.562111 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 12:03:03.563425 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 12:03:03.563425 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:03:03.563425 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:03:03.563425 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:03:03.563425 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:03:03.563425 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:03:03.563425 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:03:03.563425 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:03:03.563425 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 12:03:03.563425 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:03:03.563425 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:03:03.563425 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:03:03.563425 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:03:03.563425 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:03:03.563425 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:03:03.563425 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:03:03.563425 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:03:03.564911 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 12:03:03.564911 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 12:03:03.564911 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 12:03:03.564911 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 12:03:03.564911 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 12:03:03.564911 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 12:03:03.564911 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:03:03.564911 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:03:03.564911 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 603/1217] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 9 12:03:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:03:03.569879 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae70095c-b058-4667-ac09-cecff57c6e2b None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:03:03.678051 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae70095c-b058-4667-ac09-cecff57c6e2b None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:03:03.703138 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:03:03.716714 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb2c6d3-12a6-4524-8e5c-070786289e28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:03:03.717314 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb2c6d3-12a6-4524-8e5c-070786289e28 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:03:03.717627 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb2c6d3-12a6-4524-8e5c-070786289e28 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:03:03.717896 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb2c6d3-12a6-4524-8e5c-070786289e28 None admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:03:03.718579 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb2c6d3-12a6-4524-8e5c-070786289e28 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:03:03.718873 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb2c6d3-12a6-4524-8e5c-070786289e28 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:03:03.720479 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb2c6d3-12a6-4524-8e5c-070786289e28 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:03:03.734195 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 615/1218] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Sep 9 12:03:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:03:03.740410 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baac8a4d-6402-4918-9da7-858942cde286 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:03:03.845028 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baac8a4d-6402-4918-9da7-858942cde286 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:03:03.878673 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:03:03.885535 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee3848d-4b89-4adc-b155-1b6343b4b8b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:03:03.886186 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee3848d-4b89-4adc-b155-1b6343b4b8b4 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:03:03.886467 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee3848d-4b89-4adc-b155-1b6343b4b8b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:03:03.886729 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee3848d-4b89-4adc-b155-1b6343b4b8b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:03:03.887503 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee3848d-4b89-4adc-b155-1b6343b4b8b4 None admin] RBAC: Adding query filter params (role.id=8abeff2cb87d4b7fa72b27b028005877, scope.project.id=fe863fcd75f1406ebbffc5016078f73e, user.id=d3adb2f0583c4ff782049dacdb3ee556) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:03:03.887788 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee3848d-4b89-4adc-b155-1b6343b4b8b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:03:03.889291 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee3848d-4b89-4adc-b155-1b6343b4b8b4 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:03:03.895324 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 604/1219] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Sep 9 12:03:03 2022] GET /identity/v3/role_assignments?user.id=d3adb2f0583c4ff782049dacdb3ee556&scope.project.id=fe863fcd75f1406ebbffc5016078f73e&role.id=8abeff2cb87d4b7fa72b27b028005877 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:03:06.079342 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5933c3-467e-4faa-9c55-bdfbe5ef24fc None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:03:06.079904 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5933c3-467e-4faa-9c55-bdfbe5ef24fc None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:03:06.079904 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5933c3-467e-4faa-9c55-bdfbe5ef24fc None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:03:06.081143 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 616/1220] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:03:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:03:06.087471 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405011b8-8ef0-4f34-9f54-209291901e92 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:03:06.087471 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405011b8-8ef0-4f34-9f54-209291901e92 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:03:06.087471 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405011b8-8ef0-4f34-9f54-209291901e92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:03:06.107225 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-405011b8-8ef0-4f34-9f54-209291901e92 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:03:06.108476 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-405011b8-8ef0-4f34-9f54-209291901e92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:03:06.188292 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 605/1221] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:03:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 12:03:06.488214 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5933c3-467e-4faa-9c55-bdfbe5ef24fc None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:03:06.605082 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5933c3-467e-4faa-9c55-bdfbe5ef24fc None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:03:06.646428 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2056ed2-10a7-460a-8e7e-ab9ead11c5da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:03:06.647175 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2056ed2-10a7-460a-8e7e-ab9ead11c5da None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:03:06.647175 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2056ed2-10a7-460a-8e7e-ab9ead11c5da None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:03:06.647175 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2056ed2-10a7-460a-8e7e-ab9ead11c5da None admin] PATH_INFO: `/v3/roles/service` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:03:06.660308 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2056ed2-10a7-460a-8e7e-ab9ead11c5da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:03:06.666791 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-c2056ed2-10a7-460a-8e7e-ab9ead11c5da None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 12:03:06.666791 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:03:06.666791 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:03:06.666791 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:03:06.666791 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:03:06.666791 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:03:06.666791 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:03:06.666791 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:03:06.666791 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:03:06.666791 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:03:06.666791 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:03:06.666791 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:03:06.666791 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:03:06.666791 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:03:06.666791 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:03:06.666791 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:03:06.668218 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:03:06.668218 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:03:06.668218 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:03:06.668218 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:03:06.668218 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:03:06.668218 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:03:06.668218 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:03:06.668218 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:03:06.668218 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:03:06.668218 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:03:06.668218 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:03:06.668218 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 12:03:06.668218 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:03:06.668218 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:03:06.668218 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:03:06.668218 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:03:06.669319 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:03:06.669319 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:03:06.669319 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:03:06.669319 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:03:06.669319 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:03:06.669319 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:03:06.669319 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:03:06.669319 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:03:06.669319 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:03:06.669319 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:03:06.669319 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 12:03:06.669319 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:03:06.669319 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 617/1222] 10.222.0.49 () {56 vars in 1102 bytes} [Fri Sep 9 12:03:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:03:06.674583 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405011b8-8ef0-4f34-9f54-209291901e92 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:03:06.785556 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405011b8-8ef0-4f34-9f54-209291901e92 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:03:06.812877 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:03:06.827543 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7692a210-de4c-4a02-8a47-55a12a07aeac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:03:06.828487 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7692a210-de4c-4a02-8a47-55a12a07aeac None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:03:06.828832 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7692a210-de4c-4a02-8a47-55a12a07aeac None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:03:06.829253 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7692a210-de4c-4a02-8a47-55a12a07aeac None admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:03:06.829981 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7692a210-de4c-4a02-8a47-55a12a07aeac None admin] RBAC: Adding query filter params (name=service) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:03:06.830311 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7692a210-de4c-4a02-8a47-55a12a07aeac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:03:06.831973 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7692a210-de4c-4a02-8a47-55a12a07aeac None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:03:06.842331 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 606/1223] 10.222.0.49 () {56 vars in 1103 bytes} [Fri Sep 9 12:03:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:03:06.848147 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2056ed2-10a7-460a-8e7e-ab9ead11c5da None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:03:06.896643 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:03:06.957314 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2056ed2-10a7-460a-8e7e-ab9ead11c5da None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:03:06.997499 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57446273-e4e1-4672-8056-1968b342dffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:03:06.998133 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57446273-e4e1-4672-8056-1968b342dffc None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:03:06.998411 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57446273-e4e1-4672-8056-1968b342dffc None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:03:06.998683 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57446273-e4e1-4672-8056-1968b342dffc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:03:07.009019 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57446273-e4e1-4672-8056-1968b342dffc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:03:07.010496 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57446273-e4e1-4672-8056-1968b342dffc None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:03:07.025724 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-57446273-e4e1-4672-8056-1968b342dffc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:03:07.025724 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:03:07.025724 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:03:07.025724 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:03:07.025724 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:03:07.025724 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:03:07.025724 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:03:07.025724 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:03:07.025724 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:03:07.025724 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:03:07.025724 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:03:07.025724 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:03:07.025724 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:03:07.025724 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:03:07.025724 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:03:07.025724 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:03:07.026882 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:03:07.026882 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:03:07.026882 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:03:07.026882 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:03:07.026882 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:03:07.026882 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:03:07.026882 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:03:07.026882 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 12:03:07.026882 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:03:07.026882 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:03:07.026882 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:03:07.026882 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:03:07.026882 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:03:07.026882 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:03:07.026882 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:03:07.026882 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:03:07.026882 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:03:07.027995 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:03:07.027995 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:03:07.027995 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:03:07.031042 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 618/1224] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 12:03:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:03:07.032701 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7692a210-de4c-4a02-8a47-55a12a07aeac None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:03:07.146337 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7692a210-de4c-4a02-8a47-55a12a07aeac None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:03:07.186472 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d012d7-d6f1-41d4-b9ed-be70e179e1b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:03:07.187136 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d012d7-d6f1-41d4-b9ed-be70e179e1b5 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:03:07.187418 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d012d7-d6f1-41d4-b9ed-be70e179e1b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:03:07.187688 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d012d7-d6f1-41d4-b9ed-be70e179e1b5 None admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:03:07.188401 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d012d7-d6f1-41d4-b9ed-be70e179e1b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:03:07.188706 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d012d7-d6f1-41d4-b9ed-be70e179e1b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:03:07.190150 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d012d7-d6f1-41d4-b9ed-be70e179e1b5 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:03:07.202312 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 607/1225] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 9 12:03:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:03:07.208138 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57446273-e4e1-4672-8056-1968b342dffc None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:03:07.311899 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:03:07.319833 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57446273-e4e1-4672-8056-1968b342dffc None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:03:07.357822 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565a1324-c9b0-4b3a-977a-62b73c9d6162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:03:07.358507 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565a1324-c9b0-4b3a-977a-62b73c9d6162 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:03:07.358802 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565a1324-c9b0-4b3a-977a-62b73c9d6162 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:03:07.359112 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565a1324-c9b0-4b3a-977a-62b73c9d6162 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:03:07.366607 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565a1324-c9b0-4b3a-977a-62b73c9d6162 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:03:07.368070 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565a1324-c9b0-4b3a-977a-62b73c9d6162 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:03:07.379882 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-565a1324-c9b0-4b3a-977a-62b73c9d6162 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 09 12:03:07.379882 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:03:07.379882 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:03:07.379882 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:03:07.379882 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:03:07.379882 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:03:07.379882 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:03:07.379882 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:03:07.379882 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:03:07.379882 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:03:07.379882 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:03:07.379882 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:03:07.379882 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 12:03:07.379882 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 12:03:07.379882 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 12:03:07.379882 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 12:03:07.381059 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:03:07.381059 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:03:07.381059 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 12:03:07.381059 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:03:07.381059 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 12:03:07.381059 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:03:07.381059 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:03:07.381059 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:03:07.381059 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:03:07.381059 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:03:07.381059 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:03:07.381059 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 12:03:07.381059 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:03:07.381059 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:03:07.381059 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:03:07.381059 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:03:07.381059 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:03:07.381059 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:03:07.382367 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:03:07.382367 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:03:07.382367 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 12:03:07.382367 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 12:03:07.382367 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 12:03:07.382367 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 12:03:07.382367 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 12:03:07.382367 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 12:03:07.382367 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 09 12:03:07.382367 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:03:07.382367 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 619/1226] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 12:03:07 2022] GET /identity/v3/users/placement => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 12:03:07.388325 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d012d7-d6f1-41d4-b9ed-be70e179e1b5 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:03:07.439215 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:03:07.498624 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d012d7-d6f1-41d4-b9ed-be70e179e1b5 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:03:07.543056 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6863b6-ac29-4990-8a6a-5933fec6c984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:03:07.543707 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6863b6-ac29-4990-8a6a-5933fec6c984 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:03:07.543988 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6863b6-ac29-4990-8a6a-5933fec6c984 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:03:07.544257 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6863b6-ac29-4990-8a6a-5933fec6c984 None admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:03:07.544987 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6863b6-ac29-4990-8a6a-5933fec6c984 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:03:07.545287 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6863b6-ac29-4990-8a6a-5933fec6c984 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:03:07.546730 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6863b6-ac29-4990-8a6a-5933fec6c984 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:03:07.564625 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 608/1227] 10.222.0.49 () {56 vars in 1143 bytes} [Fri Sep 9 12:03:07 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:03:07.570265 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565a1324-c9b0-4b3a-977a-62b73c9d6162 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:03:07.688236 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565a1324-c9b0-4b3a-977a-62b73c9d6162 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:03:07.725611 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b42c46-75cf-4792-af93-9b96f0f5d54e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:03:07.726295 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b42c46-75cf-4792-af93-9b96f0f5d54e None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:03:07.726758 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b42c46-75cf-4792-af93-9b96f0f5d54e None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:03:07.727090 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b42c46-75cf-4792-af93-9b96f0f5d54e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:03:07.741250 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b42c46-75cf-4792-af93-9b96f0f5d54e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:03:07.742735 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b42c46-75cf-4792-af93-9b96f0f5d54e None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:03:07.759075 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-a4b42c46-75cf-4792-af93-9b96f0f5d54e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:03:07.759075 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:03:07.759075 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:03:07.759075 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:03:07.759075 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:03:07.759075 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:03:07.759075 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:03:07.759075 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:03:07.759075 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:03:07.759075 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:03:07.759075 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:03:07.759075 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:03:07.759075 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:03:07.759075 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:03:07.759075 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:03:07.759075 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:03:07.760193 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:03:07.760193 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:03:07.760193 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:03:07.760193 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:03:07.760193 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:03:07.760193 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:03:07.760193 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:03:07.760193 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 12:03:07.760193 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:03:07.760193 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:03:07.760193 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:03:07.760193 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:03:07.760193 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:03:07.760193 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:03:07.760193 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:03:07.760193 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:03:07.760193 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:03:07.761843 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:03:07.761843 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:03:07.761843 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:03:07.761843 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 620/1228] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 12:03:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:03:07.766529 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6863b6-ac29-4990-8a6a-5933fec6c984 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:03:07.878526 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6863b6-ac29-4990-8a6a-5933fec6c984 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:03:07.880963 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:03:07.917356 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66544413-ac9e-4d49-9ef4-0eea3a6d5586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:03:07.917978 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66544413-ac9e-4d49-9ef4-0eea3a6d5586 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:03:07.918256 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66544413-ac9e-4d49-9ef4-0eea3a6d5586 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:03:07.918528 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66544413-ac9e-4d49-9ef4-0eea3a6d5586 None admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:03:07.919326 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66544413-ac9e-4d49-9ef4-0eea3a6d5586 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:03:07.919689 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66544413-ac9e-4d49-9ef4-0eea3a6d5586 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:03:07.921297 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66544413-ac9e-4d49-9ef4-0eea3a6d5586 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:03:07.935018 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 609/1229] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 9 12:03:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:03:07.940972 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b42c46-75cf-4792-af93-9b96f0f5d54e None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:03:07.951861 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:03:08.055460 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b42c46-75cf-4792-af93-9b96f0f5d54e None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:03:08.093090 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a21424-23d3-4419-a5f2-831f64e74982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:03:08.093818 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a21424-23d3-4419-a5f2-831f64e74982 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:03:08.094093 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a21424-23d3-4419-a5f2-831f64e74982 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:03:08.094503 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a21424-23d3-4419-a5f2-831f64e74982 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:03:08.103801 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a21424-23d3-4419-a5f2-831f64e74982 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:03:08.105782 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a21424-23d3-4419-a5f2-831f64e74982 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:03:08.121498 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-71a21424-23d3-4419-a5f2-831f64e74982 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:03:08.121498 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:03:08.121498 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:03:08.121498 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:03:08.121498 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:03:08.121498 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:03:08.121498 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:03:08.121498 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:03:08.121498 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:03:08.121498 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:03:08.121498 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:03:08.121498 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:03:08.121498 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 12:03:08.121498 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 12:03:08.121498 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 12:03:08.122467 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 12:03:08.122467 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:03:08.122467 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:03:08.122467 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:03:08.122467 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:03:08.122467 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:03:08.122467 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:03:08.122467 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:03:08.122467 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 12:03:08.122467 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:03:08.122467 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:03:08.122467 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:03:08.122467 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:03:08.122467 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:03:08.122467 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:03:08.122467 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:03:08.122467 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:03:08.123569 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 12:03:08.123569 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 12:03:08.123569 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 12:03:08.123569 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 12:03:08.123569 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 12:03:08.123569 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 12:03:08.123569 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:03:08.123569 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:03:08.123569 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 621/1230] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 9 12:03:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:03:08.128876 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66544413-ac9e-4d49-9ef4-0eea3a6d5586 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:03:08.236294 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66544413-ac9e-4d49-9ef4-0eea3a6d5586 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:03:08.270671 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d1cd09-4b7c-446a-a65f-63230e3dd140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:03:08.271339 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d1cd09-4b7c-446a-a65f-63230e3dd140 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:03:08.271616 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d1cd09-4b7c-446a-a65f-63230e3dd140 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:03:08.271887 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d1cd09-4b7c-446a-a65f-63230e3dd140 None admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:03:08.272647 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d1cd09-4b7c-446a-a65f-63230e3dd140 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:03:08.272876 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d1cd09-4b7c-446a-a65f-63230e3dd140 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:03:08.274329 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d1cd09-4b7c-446a-a65f-63230e3dd140 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:03:08.287314 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 610/1231] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Sep 9 12:03:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:03:08.294055 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a21424-23d3-4419-a5f2-831f64e74982 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:03:08.368683 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:03:08.408498 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a21424-23d3-4419-a5f2-831f64e74982 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:03:08.442687 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c9e15f-e5ec-4ca0-8bbe-15783fd76a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:03:08.443333 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c9e15f-e5ec-4ca0-8bbe-15783fd76a43 None admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:03:08.443619 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c9e15f-e5ec-4ca0-8bbe-15783fd76a43 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:03:08.443885 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c9e15f-e5ec-4ca0-8bbe-15783fd76a43 None admin] PATH_INFO: `/v3/projects/fe863fcd75f1406ebbffc5016078f73e/users/d3adb2f0583c4ff782049dacdb3ee556/roles/8abeff2cb87d4b7fa72b27b028005877` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:03:08.459140 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c9e15f-e5ec-4ca0-8bbe-15783fd76a43 None admin] RBAC: Authorizing `identity:create_grant(project_id=fe863fcd75f1406ebbffc5016078f73e, user_id=d3adb2f0583c4ff782049dacdb3ee556, role_id=8abeff2cb87d4b7fa72b27b028005877)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:03:08.460570 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c9e15f-e5ec-4ca0-8bbe-15783fd76a43 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:03:08.473661 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 622/1232] 10.222.0.49 () {58 vars in 1439 bytes} [Fri Sep 9 12:03:08 2022] PUT /identity/v3/projects/fe863fcd75f1406ebbffc5016078f73e/users/d3adb2f0583c4ff782049dacdb3ee556/roles/8abeff2cb87d4b7fa72b27b028005877 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:03:10.714924 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e0edd9-81d5-423f-a41d-d556fe75089b None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:03:10.715499 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e0edd9-81d5-423f-a41d-d556fe75089b None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:03:10.715499 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e0edd9-81d5-423f-a41d-d556fe75089b None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:03:10.716994 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 611/1233] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:03:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 12:03:10.722396 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339f2fb1-06e7-4e34-b56b-da5c243929f9 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:03:10.723299 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339f2fb1-06e7-4e34-b56b-da5c243929f9 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:03:10.723299 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339f2fb1-06e7-4e34-b56b-da5c243929f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:03:10.741771 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-339f2fb1-06e7-4e34-b56b-da5c243929f9 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:03:10.742979 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-339f2fb1-06e7-4e34-b56b-da5c243929f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:03:10.817457 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 623/1234] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:03:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 12:03:11.085560 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e0edd9-81d5-423f-a41d-d556fe75089b None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:03:11.114681 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:03:11.206507 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e0edd9-81d5-423f-a41d-d556fe75089b None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:03:11.243160 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b668bb6-5845-448a-8bce-3ddc730a0a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:03:11.244405 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b668bb6-5845-448a-8bce-3ddc730a0a64 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:03:11.244677 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b668bb6-5845-448a-8bce-3ddc730a0a64 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:03:11.244943 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b668bb6-5845-448a-8bce-3ddc730a0a64 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:03:11.256253 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b668bb6-5845-448a-8bce-3ddc730a0a64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:03:11.262215 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-5b668bb6-5845-448a-8bce-3ddc730a0a64 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 12:03:11.262215 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:03:11.262215 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:03:11.262215 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:03:11.262215 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:03:11.262215 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:03:11.262215 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:03:11.262215 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:03:11.262215 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:03:11.262215 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:03:11.262215 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:03:11.262215 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:03:11.262215 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:03:11.262215 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:03:11.262215 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:03:11.262215 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:03:11.263460 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:03:11.263460 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:03:11.263460 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:03:11.263460 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:03:11.263460 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:03:11.263460 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:03:11.263460 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:03:11.263460 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:03:11.263460 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:03:11.263460 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:03:11.263460 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:03:11.263460 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 12:03:11.263460 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:03:11.263460 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:03:11.263460 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:03:11.263460 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:03:11.264672 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:03:11.264672 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:03:11.264672 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:03:11.264672 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:03:11.264672 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:03:11.264672 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:03:11.264672 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:03:11.264672 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:03:11.264672 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:03:11.264672 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:03:11.264672 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 12:03:11.264672 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:03:11.264672 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 612/1235] 10.222.0.49 () {56 vars in 1102 bytes} [Fri Sep 9 12:03:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 12:03:11.270168 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339f2fb1-06e7-4e34-b56b-da5c243929f9 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:03:11.323395 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:03:11.382640 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339f2fb1-06e7-4e34-b56b-da5c243929f9 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:03:11.418699 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6bb8ad-8e50-453f-a75e-12e50722a96b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:03:11.419344 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6bb8ad-8e50-453f-a75e-12e50722a96b None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:03:11.419620 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6bb8ad-8e50-453f-a75e-12e50722a96b None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:03:11.419894 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6bb8ad-8e50-453f-a75e-12e50722a96b None admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:03:11.420652 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6bb8ad-8e50-453f-a75e-12e50722a96b None admin] RBAC: Adding query filter params (name=service) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:03:11.420957 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6bb8ad-8e50-453f-a75e-12e50722a96b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:03:11.422367 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6bb8ad-8e50-453f-a75e-12e50722a96b None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:03:11.431347 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 624/1236] 10.222.0.49 () {56 vars in 1103 bytes} [Fri Sep 9 12:03:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:03:11.437663 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b668bb6-5845-448a-8bce-3ddc730a0a64 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:03:11.529563 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:03:11.547514 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b668bb6-5845-448a-8bce-3ddc730a0a64 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:03:11.584422 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3e4de6-89a9-40fe-85d0-daac77f71dfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:03:11.585025 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3e4de6-89a9-40fe-85d0-daac77f71dfb None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:03:11.585301 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3e4de6-89a9-40fe-85d0-daac77f71dfb None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:03:11.585562 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3e4de6-89a9-40fe-85d0-daac77f71dfb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:03:11.596397 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3e4de6-89a9-40fe-85d0-daac77f71dfb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:03:11.598032 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3e4de6-89a9-40fe-85d0-daac77f71dfb None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:03:11.614274 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-8e3e4de6-89a9-40fe-85d0-daac77f71dfb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:03:11.614274 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:03:11.614274 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:03:11.614274 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:03:11.614274 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:03:11.614274 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:03:11.614274 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:03:11.614274 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:03:11.614274 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:03:11.614274 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:03:11.614274 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:03:11.614274 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:03:11.614274 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:03:11.614274 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:03:11.614274 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:03:11.614274 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:03:11.615401 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:03:11.615401 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:03:11.615401 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:03:11.615401 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:03:11.615401 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:03:11.615401 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:03:11.615401 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:03:11.615401 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 12:03:11.615401 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:03:11.615401 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:03:11.615401 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:03:11.615401 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:03:11.615401 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:03:11.615401 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:03:11.615401 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:03:11.615401 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:03:11.615401 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:03:11.616332 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:03:11.616332 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:03:11.616332 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:03:11.616332 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 613/1237] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 12:03:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:03:11.621854 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6bb8ad-8e50-453f-a75e-12e50722a96b None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:03:11.731309 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6bb8ad-8e50-453f-a75e-12e50722a96b None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:03:11.741478 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:03:11.770503 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbdcd9a-8a47-47c8-b2e5-fa0725402a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:03:11.771223 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbdcd9a-8a47-47c8-b2e5-fa0725402a54 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:03:11.771520 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbdcd9a-8a47-47c8-b2e5-fa0725402a54 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:03:11.771790 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbdcd9a-8a47-47c8-b2e5-fa0725402a54 None admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:03:11.772560 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbdcd9a-8a47-47c8-b2e5-fa0725402a54 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:03:11.772902 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbdcd9a-8a47-47c8-b2e5-fa0725402a54 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:03:11.774310 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbdcd9a-8a47-47c8-b2e5-fa0725402a54 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:03:11.787140 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 625/1238] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 9 12:03:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:03:11.793091 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3e4de6-89a9-40fe-85d0-daac77f71dfb None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:03:11.909614 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3e4de6-89a9-40fe-85d0-daac77f71dfb None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:03:11.948758 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2064db3e-77ea-409c-bf01-6e13e999bb20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:03:11.949397 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2064db3e-77ea-409c-bf01-6e13e999bb20 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:03:11.949671 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2064db3e-77ea-409c-bf01-6e13e999bb20 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:03:11.949938 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2064db3e-77ea-409c-bf01-6e13e999bb20 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:03:11.952761 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:03:11.958658 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2064db3e-77ea-409c-bf01-6e13e999bb20 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:03:11.960454 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2064db3e-77ea-409c-bf01-6e13e999bb20 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:03:11.973440 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-2064db3e-77ea-409c-bf01-6e13e999bb20 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 09 12:03:11.973440 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:03:11.973440 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:03:11.973440 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:03:11.973440 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:03:11.973440 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:03:11.973440 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:03:11.973440 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:03:11.973440 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:03:11.973440 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:03:11.973440 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:03:11.973440 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:03:11.973440 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 12:03:11.973440 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 12:03:11.973440 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 12:03:11.973440 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 12:03:11.980072 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:03:11.980072 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:03:11.980072 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 12:03:11.980072 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:03:11.980072 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 12:03:11.980072 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:03:11.980072 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:03:11.980072 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:03:11.980072 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:03:11.980072 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:03:11.980072 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:03:11.980072 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 12:03:11.980072 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:03:11.980072 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:03:11.980072 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:03:11.980072 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:03:11.980072 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:03:11.980072 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:03:11.981292 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:03:11.981292 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:03:11.981292 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 12:03:11.981292 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 12:03:11.981292 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 12:03:11.981292 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 12:03:11.981292 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 12:03:11.981292 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 12:03:11.981292 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 09 12:03:11.981292 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:03:11.981292 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 614/1239] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 12:03:11 2022] GET /identity/v3/users/placement => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 12:03:11.982040 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbdcd9a-8a47-47c8-b2e5-fa0725402a54 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:03:12.089629 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbdcd9a-8a47-47c8-b2e5-fa0725402a54 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:03:12.128033 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0046e6a1-20e6-475f-bc19-edc31e62ff2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:03:12.128707 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0046e6a1-20e6-475f-bc19-edc31e62ff2d None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:03:12.128962 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0046e6a1-20e6-475f-bc19-edc31e62ff2d None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:03:12.129276 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0046e6a1-20e6-475f-bc19-edc31e62ff2d None admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:03:12.129960 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0046e6a1-20e6-475f-bc19-edc31e62ff2d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:03:12.130244 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0046e6a1-20e6-475f-bc19-edc31e62ff2d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:03:12.131772 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0046e6a1-20e6-475f-bc19-edc31e62ff2d None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:03:12.157188 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 626/1240] 10.222.0.49 () {56 vars in 1143 bytes} [Fri Sep 9 12:03:11 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:03:12.158534 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2064db3e-77ea-409c-bf01-6e13e999bb20 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:03:12.267759 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2064db3e-77ea-409c-bf01-6e13e999bb20 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:03:12.306844 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92180f8-9fef-468e-8777-28035250e6e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:03:12.307495 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92180f8-9fef-468e-8777-28035250e6e1 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:03:12.307774 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92180f8-9fef-468e-8777-28035250e6e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:03:12.308039 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92180f8-9fef-468e-8777-28035250e6e1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:03:12.373559 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:03:12.381239 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92180f8-9fef-468e-8777-28035250e6e1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:03:12.382862 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92180f8-9fef-468e-8777-28035250e6e1 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:03:12.486228 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 615/1241] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 9 12:03:12 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 12:03:12.492075 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0046e6a1-20e6-475f-bc19-edc31e62ff2d None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:03:12.521059 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:03:12.606611 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0046e6a1-20e6-475f-bc19-edc31e62ff2d None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:03:12.646421 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960a866e-c902-41d4-be06-c38ad7003854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:03:12.646846 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960a866e-c902-41d4-be06-c38ad7003854 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:03:12.647181 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960a866e-c902-41d4-be06-c38ad7003854 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:03:12.647459 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960a866e-c902-41d4-be06-c38ad7003854 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:03:12.658491 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960a866e-c902-41d4-be06-c38ad7003854 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:03:12.659964 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960a866e-c902-41d4-be06-c38ad7003854 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:03:12.676258 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-960a866e-c902-41d4-be06-c38ad7003854 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:03:12.676258 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:03:12.676258 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:03:12.676258 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:03:12.676258 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:03:12.676258 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:03:12.676258 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:03:12.676258 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:03:12.676258 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:03:12.676258 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:03:12.676258 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:03:12.676258 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:03:12.676258 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:03:12.676258 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:03:12.676258 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:03:12.676258 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:03:12.677620 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:03:12.677620 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:03:12.677620 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:03:12.677620 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:03:12.677620 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:03:12.677620 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:03:12.677620 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:03:12.677620 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 12:03:12.677620 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:03:12.677620 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:03:12.677620 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:03:12.677620 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:03:12.677620 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:03:12.677620 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:03:12.677620 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:03:12.677620 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:03:12.677620 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:03:12.678970 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:03:12.678970 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:03:12.678970 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:03:12.678970 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 627/1242] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 12:03:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:03:12.683315 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92180f8-9fef-468e-8777-28035250e6e1 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:03:12.794691 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:03:12.801080 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92180f8-9fef-468e-8777-28035250e6e1 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:03:12.840311 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96764628-c86c-4e30-8f50-1ee2d719f6d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:03:12.840953 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96764628-c86c-4e30-8f50-1ee2d719f6d4 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:03:12.841235 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96764628-c86c-4e30-8f50-1ee2d719f6d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:03:12.841495 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96764628-c86c-4e30-8f50-1ee2d719f6d4 None admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:03:12.842181 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96764628-c86c-4e30-8f50-1ee2d719f6d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:03:12.842463 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96764628-c86c-4e30-8f50-1ee2d719f6d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:03:12.843978 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96764628-c86c-4e30-8f50-1ee2d719f6d4 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:03:12.856824 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 616/1243] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 9 12:03:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:03:12.862662 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960a866e-c902-41d4-be06-c38ad7003854 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:03:12.950333 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:03:12.977136 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960a866e-c902-41d4-be06-c38ad7003854 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:03:13.015083 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e580d42-ebe5-4023-8ab3-ec728add5ce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:03:13.016019 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e580d42-ebe5-4023-8ab3-ec728add5ce2 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:03:13.016294 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e580d42-ebe5-4023-8ab3-ec728add5ce2 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:03:13.016659 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e580d42-ebe5-4023-8ab3-ec728add5ce2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:03:13.025228 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e580d42-ebe5-4023-8ab3-ec728add5ce2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:03:13.026930 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e580d42-ebe5-4023-8ab3-ec728add5ce2 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:03:13.039921 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-1e580d42-ebe5-4023-8ab3-ec728add5ce2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:03:13.039921 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:03:13.039921 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:03:13.039921 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:03:13.039921 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:03:13.039921 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:03:13.039921 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:03:13.039921 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:03:13.039921 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:03:13.039921 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:03:13.039921 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:03:13.039921 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:03:13.039921 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 12:03:13.039921 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 12:03:13.039921 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 12:03:13.041623 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 12:03:13.041623 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:03:13.041623 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:03:13.041623 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:03:13.041623 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:03:13.041623 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:03:13.041623 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:03:13.041623 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:03:13.041623 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 12:03:13.041623 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:03:13.041623 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:03:13.041623 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:03:13.041623 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:03:13.041623 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:03:13.041623 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:03:13.041623 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:03:13.041623 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:03:13.042801 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 12:03:13.042801 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 12:03:13.042801 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 12:03:13.042801 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 12:03:13.042801 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 12:03:13.042801 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 12:03:13.042801 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:03:13.042801 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:03:13.044239 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 628/1244] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 9 12:03:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:03:13.050394 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96764628-c86c-4e30-8f50-1ee2d719f6d4 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:03:13.172876 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96764628-c86c-4e30-8f50-1ee2d719f6d4 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:03:13.211310 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec02c77-199a-42b5-8f53-be675d49b89e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:03:13.212535 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec02c77-199a-42b5-8f53-be675d49b89e None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:03:13.212920 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec02c77-199a-42b5-8f53-be675d49b89e None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:03:13.213197 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec02c77-199a-42b5-8f53-be675d49b89e None admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:03:13.213890 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec02c77-199a-42b5-8f53-be675d49b89e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:03:13.214198 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec02c77-199a-42b5-8f53-be675d49b89e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:03:13.215702 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec02c77-199a-42b5-8f53-be675d49b89e None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:03:13.229546 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 617/1245] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Sep 9 12:03:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:03:13.235851 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e580d42-ebe5-4023-8ab3-ec728add5ce2 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:03:13.344378 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e580d42-ebe5-4023-8ab3-ec728add5ce2 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:03:13.380761 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5698ccbd-4c4b-4b50-b1a5-1b58538a4250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:03:13.381667 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5698ccbd-4c4b-4b50-b1a5-1b58538a4250 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:03:13.382012 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5698ccbd-4c4b-4b50-b1a5-1b58538a4250 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:03:13.382280 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5698ccbd-4c4b-4b50-b1a5-1b58538a4250 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:03:13.383247 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5698ccbd-4c4b-4b50-b1a5-1b58538a4250 None admin] RBAC: Adding query filter params (role.id=8abeff2cb87d4b7fa72b27b028005877, scope.project.id=fe863fcd75f1406ebbffc5016078f73e, user.id=d3adb2f0583c4ff782049dacdb3ee556) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:03:13.383585 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5698ccbd-4c4b-4b50-b1a5-1b58538a4250 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:03:13.385309 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5698ccbd-4c4b-4b50-b1a5-1b58538a4250 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:03:13.392635 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 629/1246] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Sep 9 12:03:13 2022] GET /identity/v3/role_assignments?user.id=d3adb2f0583c4ff782049dacdb3ee556&scope.project.id=fe863fcd75f1406ebbffc5016078f73e&role.id=8abeff2cb87d4b7fa72b27b028005877 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:03:15.839773 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145e2eae-1b34-42ed-8c29-d014ca12e618 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:03:15.840320 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145e2eae-1b34-42ed-8c29-d014ca12e618 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:03:15.840320 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145e2eae-1b34-42ed-8c29-d014ca12e618 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:03:15.841652 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 618/1247] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:03:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:03:15.847433 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7313d418-a2bf-406c-89ee-6ad4df5a8120 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:03:15.847905 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7313d418-a2bf-406c-89ee-6ad4df5a8120 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:03:15.848003 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7313d418-a2bf-406c-89ee-6ad4df5a8120 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:03:15.855659 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:03:15.869982 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-7313d418-a2bf-406c-89ee-6ad4df5a8120 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:03:15.871285 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-7313d418-a2bf-406c-89ee-6ad4df5a8120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:03:15.944234 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 630/1248] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:03:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 12:03:16.277791 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145e2eae-1b34-42ed-8c29-d014ca12e618 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:03:16.297246 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:03:16.388328 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145e2eae-1b34-42ed-8c29-d014ca12e618 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:03:16.423246 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f12bc9-237c-4b3c-9d24-c4239d045856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:03:16.423873 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f12bc9-237c-4b3c-9d24-c4239d045856 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:03:16.424145 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f12bc9-237c-4b3c-9d24-c4239d045856 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:03:16.424411 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f12bc9-237c-4b3c-9d24-c4239d045856 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:03:16.436918 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f12bc9-237c-4b3c-9d24-c4239d045856 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:03:16.441990 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-c7f12bc9-237c-4b3c-9d24-c4239d045856 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 12:03:16.441990 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:03:16.441990 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:03:16.441990 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:03:16.441990 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:03:16.441990 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:03:16.441990 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:03:16.441990 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:03:16.441990 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:03:16.441990 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:03:16.441990 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:03:16.441990 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:03:16.441990 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:03:16.441990 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:03:16.441990 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:03:16.441990 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:03:16.443166 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:03:16.443166 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:03:16.443166 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:03:16.443166 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:03:16.443166 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:03:16.443166 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:03:16.443166 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:03:16.443166 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:03:16.443166 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:03:16.443166 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:03:16.443166 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:03:16.443166 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 12:03:16.443166 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:03:16.443166 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:03:16.443166 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:03:16.443166 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:03:16.444236 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:03:16.444236 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:03:16.444236 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:03:16.444236 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:03:16.444236 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:03:16.444236 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:03:16.444236 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:03:16.444236 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:03:16.444236 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:03:16.444236 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:03:16.444236 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 12:03:16.444236 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:03:16.444236 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 619/1249] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Sep 9 12:03:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:03:16.450954 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7313d418-a2bf-406c-89ee-6ad4df5a8120 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:03:16.558257 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7313d418-a2bf-406c-89ee-6ad4df5a8120 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:03:16.588943 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:03:16.594505 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3926af6-7654-4eab-866d-48afe9025166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:03:16.595148 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3926af6-7654-4eab-866d-48afe9025166 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:03:16.595423 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3926af6-7654-4eab-866d-48afe9025166 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:03:16.595687 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3926af6-7654-4eab-866d-48afe9025166 None admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:03:16.596363 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3926af6-7654-4eab-866d-48afe9025166 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:03:16.596660 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3926af6-7654-4eab-866d-48afe9025166 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:03:16.598009 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3926af6-7654-4eab-866d-48afe9025166 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:03:16.607089 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 631/1250] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 9 12:03:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:03:16.613399 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f12bc9-237c-4b3c-9d24-c4239d045856 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:03:16.703157 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:03:16.729681 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f12bc9-237c-4b3c-9d24-c4239d045856 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:03:16.769925 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a721d8d5-2841-4766-9a05-cfbebc904bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:03:16.770556 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a721d8d5-2841-4766-9a05-cfbebc904bdf None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:03:16.770859 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a721d8d5-2841-4766-9a05-cfbebc904bdf None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:03:16.771156 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a721d8d5-2841-4766-9a05-cfbebc904bdf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:03:16.782693 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a721d8d5-2841-4766-9a05-cfbebc904bdf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:03:16.784231 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a721d8d5-2841-4766-9a05-cfbebc904bdf None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:03:16.800379 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-a721d8d5-2841-4766-9a05-cfbebc904bdf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:03:16.800379 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:03:16.800379 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:03:16.800379 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:03:16.800379 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:03:16.800379 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:03:16.800379 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:03:16.800379 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:03:16.800379 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:03:16.800379 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:03:16.800379 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:03:16.800379 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:03:16.800379 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:03:16.800379 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:03:16.800379 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:03:16.800379 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:03:16.801596 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:03:16.801596 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:03:16.801596 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:03:16.801596 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:03:16.801596 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:03:16.801596 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:03:16.801596 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:03:16.801596 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 12:03:16.801596 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:03:16.801596 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:03:16.801596 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:03:16.801596 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:03:16.801596 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:03:16.801596 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:03:16.801596 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:03:16.801596 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:03:16.801596 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:03:16.802498 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:03:16.802498 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:03:16.802498 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:03:16.802498 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 620/1251] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 12:03:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:03:16.807990 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3926af6-7654-4eab-866d-48afe9025166 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:03:16.924330 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3926af6-7654-4eab-866d-48afe9025166 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:03:16.960609 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb10de7-2b3d-4a9f-9f9b-de4edb56b909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:03:16.961268 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb10de7-2b3d-4a9f-9f9b-de4edb56b909 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:03:16.962032 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb10de7-2b3d-4a9f-9f9b-de4edb56b909 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:03:16.962443 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb10de7-2b3d-4a9f-9f9b-de4edb56b909 None admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:03:16.963102 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb10de7-2b3d-4a9f-9f9b-de4edb56b909 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:03:16.963405 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb10de7-2b3d-4a9f-9f9b-de4edb56b909 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:03:16.964902 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb10de7-2b3d-4a9f-9f9b-de4edb56b909 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:03:16.978809 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 632/1252] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 9 12:03:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:03:16.985105 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a721d8d5-2841-4766-9a05-cfbebc904bdf None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:03:17.105397 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a721d8d5-2841-4766-9a05-cfbebc904bdf None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:03:17.140026 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:03:17.150309 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5041333d-663a-4c84-a9cf-99d6e89735d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:03:17.151031 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5041333d-663a-4c84-a9cf-99d6e89735d0 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:03:17.151416 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5041333d-663a-4c84-a9cf-99d6e89735d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:03:17.151624 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5041333d-663a-4c84-a9cf-99d6e89735d0 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:03:17.160192 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5041333d-663a-4c84-a9cf-99d6e89735d0 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:03:17.161658 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5041333d-663a-4c84-a9cf-99d6e89735d0 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:03:17.174321 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-5041333d-663a-4c84-a9cf-99d6e89735d0 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 09 12:03:17.174321 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:03:17.174321 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:03:17.174321 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:03:17.174321 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:03:17.174321 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:03:17.174321 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:03:17.174321 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:03:17.174321 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:03:17.174321 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:03:17.174321 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:03:17.174321 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:03:17.174321 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 12:03:17.174321 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 12:03:17.174321 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 12:03:17.174321 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 12:03:17.175623 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:03:17.175623 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:03:17.175623 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 12:03:17.175623 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:03:17.175623 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 12:03:17.175623 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:03:17.175623 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:03:17.175623 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:03:17.175623 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:03:17.175623 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:03:17.175623 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:03:17.175623 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 12:03:17.175623 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:03:17.175623 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:03:17.175623 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:03:17.175623 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:03:17.175623 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:03:17.175623 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:03:17.176808 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:03:17.176808 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:03:17.176808 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 12:03:17.176808 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 12:03:17.176808 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 12:03:17.176808 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 12:03:17.176808 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 12:03:17.176808 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 12:03:17.176808 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 09 12:03:17.176808 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:03:17.176808 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 621/1253] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 12:03:16 2022] GET /identity/v3/users/placement => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:03:17.181996 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb10de7-2b3d-4a9f-9f9b-de4edb56b909 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:03:17.240601 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:03:17.288685 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb10de7-2b3d-4a9f-9f9b-de4edb56b909 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:03:17.324313 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57126bad-aff8-4cc5-a4bb-49099f83ace9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:03:17.324933 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57126bad-aff8-4cc5-a4bb-49099f83ace9 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:03:17.325212 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57126bad-aff8-4cc5-a4bb-49099f83ace9 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:03:17.325479 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57126bad-aff8-4cc5-a4bb-49099f83ace9 None admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:03:17.326216 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57126bad-aff8-4cc5-a4bb-49099f83ace9 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:03:17.326517 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57126bad-aff8-4cc5-a4bb-49099f83ace9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:03:17.327992 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57126bad-aff8-4cc5-a4bb-49099f83ace9 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:03:17.346950 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 633/1254] 10.222.0.49 () {56 vars in 1143 bytes} [Fri Sep 9 12:03:17 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:03:17.354028 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5041333d-663a-4c84-a9cf-99d6e89735d0 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:03:17.460957 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5041333d-663a-4c84-a9cf-99d6e89735d0 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:03:17.498747 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c89c868-2315-4807-90dc-cdff42437a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:03:17.499688 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c89c868-2315-4807-90dc-cdff42437a22 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:03:17.499965 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c89c868-2315-4807-90dc-cdff42437a22 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:03:17.500444 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c89c868-2315-4807-90dc-cdff42437a22 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:03:17.564068 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:03:17.577136 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c89c868-2315-4807-90dc-cdff42437a22 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:03:17.578610 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c89c868-2315-4807-90dc-cdff42437a22 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:03:17.682782 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 622/1255] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 9 12:03:17 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 12:03:17.689322 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57126bad-aff8-4cc5-a4bb-49099f83ace9 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:03:17.808002 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57126bad-aff8-4cc5-a4bb-49099f83ace9 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:03:17.822294 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:03:17.844423 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d844fda-b879-415e-8b1d-08c25005880f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:03:17.845048 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d844fda-b879-415e-8b1d-08c25005880f None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:03:17.845322 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d844fda-b879-415e-8b1d-08c25005880f None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:03:17.845588 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d844fda-b879-415e-8b1d-08c25005880f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:03:17.856226 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d844fda-b879-415e-8b1d-08c25005880f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:03:17.857783 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d844fda-b879-415e-8b1d-08c25005880f None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:03:17.873240 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-5d844fda-b879-415e-8b1d-08c25005880f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:03:17.873240 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:03:17.873240 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:03:17.873240 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:03:17.873240 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:03:17.873240 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:03:17.873240 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:03:17.873240 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:03:17.873240 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:03:17.873240 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:03:17.873240 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:03:17.873240 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:03:17.873240 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:03:17.873240 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:03:17.873240 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:03:17.873240 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:03:17.874366 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:03:17.874366 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:03:17.874366 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:03:17.874366 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:03:17.874366 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:03:17.874366 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:03:17.874366 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:03:17.874366 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 12:03:17.874366 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:03:17.874366 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:03:17.874366 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:03:17.874366 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:03:17.874366 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:03:17.874366 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:03:17.874366 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:03:17.874366 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:03:17.874366 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:03:17.876214 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:03:17.876214 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:03:17.876214 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:03:17.876214 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 634/1256] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 12:03:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 12:03:17.880782 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c89c868-2315-4807-90dc-cdff42437a22 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:03:17.982989 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:03:17.997288 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c89c868-2315-4807-90dc-cdff42437a22 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:03:18.034080 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6eaed7-afa8-4f51-bb10-dea946911824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:03:18.034684 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6eaed7-afa8-4f51-bb10-dea946911824 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:03:18.034993 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6eaed7-afa8-4f51-bb10-dea946911824 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:03:18.035264 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6eaed7-afa8-4f51-bb10-dea946911824 None admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:03:18.035906 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6eaed7-afa8-4f51-bb10-dea946911824 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:03:18.036351 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6eaed7-afa8-4f51-bb10-dea946911824 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:03:18.037840 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6eaed7-afa8-4f51-bb10-dea946911824 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:03:18.050050 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 623/1257] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 9 12:03:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:03:18.062178 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d844fda-b879-415e-8b1d-08c25005880f None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:03:18.190350 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d844fda-b879-415e-8b1d-08c25005880f None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:03:18.224985 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba778360-4413-4ce4-b32a-5844531ba75d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:03:18.225618 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba778360-4413-4ce4-b32a-5844531ba75d None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:03:18.226115 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba778360-4413-4ce4-b32a-5844531ba75d None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:03:18.226629 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba778360-4413-4ce4-b32a-5844531ba75d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:03:18.237922 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba778360-4413-4ce4-b32a-5844531ba75d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:03:18.239401 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba778360-4413-4ce4-b32a-5844531ba75d None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:03:18.251762 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-ba778360-4413-4ce4-b32a-5844531ba75d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:03:18.251762 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:03:18.251762 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:03:18.251762 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:03:18.251762 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:03:18.251762 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:03:18.251762 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:03:18.251762 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:03:18.251762 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:03:18.251762 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:03:18.251762 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:03:18.251762 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:03:18.251762 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 12:03:18.251762 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 12:03:18.251762 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 12:03:18.253260 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 12:03:18.253260 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:03:18.253260 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:03:18.253260 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:03:18.253260 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:03:18.253260 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:03:18.253260 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:03:18.253260 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:03:18.253260 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 12:03:18.253260 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:03:18.253260 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:03:18.253260 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:03:18.253260 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:03:18.253260 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:03:18.253260 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:03:18.253260 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:03:18.253260 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:03:18.255020 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 12:03:18.255020 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 12:03:18.255020 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 12:03:18.255020 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 12:03:18.255020 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 12:03:18.255020 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 12:03:18.255020 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:03:18.255020 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:03:18.255020 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 635/1258] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 9 12:03:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 12:03:18.258929 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6eaed7-afa8-4f51-bb10-dea946911824 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:03:18.360823 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6eaed7-afa8-4f51-bb10-dea946911824 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:03:18.395095 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d472f075-4ffa-4673-ae37-08686a15a855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:03:18.395694 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d472f075-4ffa-4673-ae37-08686a15a855 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:03:18.395969 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d472f075-4ffa-4673-ae37-08686a15a855 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:03:18.396235 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d472f075-4ffa-4673-ae37-08686a15a855 None admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:03:18.396915 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d472f075-4ffa-4673-ae37-08686a15a855 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:03:18.397214 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d472f075-4ffa-4673-ae37-08686a15a855 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:03:18.398642 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d472f075-4ffa-4673-ae37-08686a15a855 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:03:18.410589 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 624/1259] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Sep 9 12:03:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:03:18.417338 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba778360-4413-4ce4-b32a-5844531ba75d None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:03:18.443063 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:03:18.515748 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba778360-4413-4ce4-b32a-5844531ba75d None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:03:18.548565 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ef1a3a-a94e-4ef5-a9ef-e09b9aaa0480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:03:18.549158 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ef1a3a-a94e-4ef5-a9ef-e09b9aaa0480 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:03:18.549436 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ef1a3a-a94e-4ef5-a9ef-e09b9aaa0480 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:03:18.549702 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ef1a3a-a94e-4ef5-a9ef-e09b9aaa0480 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:03:18.550408 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ef1a3a-a94e-4ef5-a9ef-e09b9aaa0480 None admin] RBAC: Adding query filter params (role.id=3f2ca33902a04b6dbc2ef7d06d7289d3, scope.project.id=fe863fcd75f1406ebbffc5016078f73e, user.id=d3adb2f0583c4ff782049dacdb3ee556) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:03:18.550709 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ef1a3a-a94e-4ef5-a9ef-e09b9aaa0480 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:03:18.552161 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ef1a3a-a94e-4ef5-a9ef-e09b9aaa0480 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:03:18.557601 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 636/1260] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Sep 9 12:03:18 2022] GET /identity/v3/role_assignments?user.id=d3adb2f0583c4ff782049dacdb3ee556&scope.project.id=fe863fcd75f1406ebbffc5016078f73e&role.id=3f2ca33902a04b6dbc2ef7d06d7289d3 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:03:20.667225 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60549606-d6e4-4b22-8e2c-17fde292cb62 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:03:20.667719 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60549606-d6e4-4b22-8e2c-17fde292cb62 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:03:20.667815 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60549606-d6e4-4b22-8e2c-17fde292cb62 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:03:20.669095 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 625/1261] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:03:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:03:20.674228 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5642124c-a809-4bbe-855b-a2573303f058 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:03:20.674622 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5642124c-a809-4bbe-855b-a2573303f058 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:03:20.674776 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5642124c-a809-4bbe-855b-a2573303f058 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:03:20.694590 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-5642124c-a809-4bbe-855b-a2573303f058 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:03:20.696496 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-5642124c-a809-4bbe-855b-a2573303f058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:03:20.760932 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 637/1262] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:03:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 12:03:21.027905 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60549606-d6e4-4b22-8e2c-17fde292cb62 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:03:21.038567 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:03:21.136872 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60549606-d6e4-4b22-8e2c-17fde292cb62 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:03:21.173004 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f396fbf7-b3ff-4482-af99-cda4399f675e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:03:21.173630 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f396fbf7-b3ff-4482-af99-cda4399f675e None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:03:21.173903 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f396fbf7-b3ff-4482-af99-cda4399f675e None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:03:21.174165 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f396fbf7-b3ff-4482-af99-cda4399f675e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:03:21.184375 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f396fbf7-b3ff-4482-af99-cda4399f675e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:03:21.189430 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-f396fbf7-b3ff-4482-af99-cda4399f675e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 12:03:21.189430 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:03:21.189430 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:03:21.189430 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:03:21.189430 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:03:21.189430 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:03:21.189430 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:03:21.189430 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:03:21.189430 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:03:21.189430 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:03:21.189430 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:03:21.189430 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:03:21.189430 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:03:21.189430 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:03:21.189430 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:03:21.189430 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:03:21.190534 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:03:21.190534 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:03:21.190534 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:03:21.190534 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:03:21.190534 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:03:21.190534 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:03:21.190534 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:03:21.190534 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:03:21.190534 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:03:21.190534 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:03:21.190534 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:03:21.190534 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 12:03:21.190534 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:03:21.190534 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:03:21.190534 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:03:21.190534 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:03:21.191615 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:03:21.191615 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:03:21.191615 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:03:21.191615 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:03:21.191615 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:03:21.191615 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:03:21.191615 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:03:21.191615 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:03:21.191615 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:03:21.191615 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:03:21.191615 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 12:03:21.191615 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:03:21.191615 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 626/1263] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Sep 9 12:03:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:03:21.197352 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5642124c-a809-4bbe-855b-a2573303f058 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:03:21.215527 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:03:21.292901 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5642124c-a809-4bbe-855b-a2573303f058 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:03:21.325783 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d224f6b-d8f3-45be-b6b1-89fce62d03c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:03:21.326371 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d224f6b-d8f3-45be-b6b1-89fce62d03c0 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:03:21.326643 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d224f6b-d8f3-45be-b6b1-89fce62d03c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:03:21.326928 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d224f6b-d8f3-45be-b6b1-89fce62d03c0 None admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:03:21.327579 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d224f6b-d8f3-45be-b6b1-89fce62d03c0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:03:21.327873 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d224f6b-d8f3-45be-b6b1-89fce62d03c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:03:21.329210 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d224f6b-d8f3-45be-b6b1-89fce62d03c0 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:03:21.337965 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 638/1264] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 9 12:03:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:03:21.343643 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f396fbf7-b3ff-4482-af99-cda4399f675e None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:03:21.411287 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:03:21.444543 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f396fbf7-b3ff-4482-af99-cda4399f675e None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:03:21.477398 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefe32a2-12d5-45fd-abbf-1005fe729aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:03:21.478004 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefe32a2-12d5-45fd-abbf-1005fe729aa6 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:03:21.478277 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefe32a2-12d5-45fd-abbf-1005fe729aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:03:21.478543 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefe32a2-12d5-45fd-abbf-1005fe729aa6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:03:21.488079 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefe32a2-12d5-45fd-abbf-1005fe729aa6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:03:21.489525 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefe32a2-12d5-45fd-abbf-1005fe729aa6 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:03:21.504480 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-cefe32a2-12d5-45fd-abbf-1005fe729aa6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:03:21.504480 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:03:21.504480 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:03:21.504480 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:03:21.504480 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:03:21.504480 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:03:21.504480 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:03:21.504480 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:03:21.504480 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:03:21.504480 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:03:21.504480 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:03:21.504480 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:03:21.504480 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:03:21.504480 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:03:21.504480 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:03:21.504480 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:03:21.505639 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:03:21.505639 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:03:21.505639 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:03:21.505639 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:03:21.505639 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:03:21.505639 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:03:21.505639 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:03:21.505639 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 12:03:21.505639 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:03:21.505639 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:03:21.505639 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:03:21.505639 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:03:21.505639 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:03:21.505639 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:03:21.505639 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:03:21.505639 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:03:21.505639 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:03:21.506532 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:03:21.506532 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:03:21.506532 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:03:21.506532 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 627/1265] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 12:03:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:03:21.512065 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d224f6b-d8f3-45be-b6b1-89fce62d03c0 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:03:21.589724 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:03:21.607846 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d224f6b-d8f3-45be-b6b1-89fce62d03c0 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:03:21.641053 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8e5aad-1463-408c-a8cf-c5cc90a2cf48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:03:21.641656 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8e5aad-1463-408c-a8cf-c5cc90a2cf48 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:03:21.641934 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8e5aad-1463-408c-a8cf-c5cc90a2cf48 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:03:21.642203 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8e5aad-1463-408c-a8cf-c5cc90a2cf48 None admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:03:21.642837 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8e5aad-1463-408c-a8cf-c5cc90a2cf48 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:03:21.643165 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8e5aad-1463-408c-a8cf-c5cc90a2cf48 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:03:21.644519 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8e5aad-1463-408c-a8cf-c5cc90a2cf48 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:03:21.656001 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 639/1266] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 9 12:03:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:03:21.661610 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefe32a2-12d5-45fd-abbf-1005fe729aa6 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:03:21.764397 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefe32a2-12d5-45fd-abbf-1005fe729aa6 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:03:21.787383 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:03:21.801679 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d914b37d-966b-4a0f-ac0b-497b49fa12cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:03:21.802292 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d914b37d-966b-4a0f-ac0b-497b49fa12cf None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:03:21.802566 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d914b37d-966b-4a0f-ac0b-497b49fa12cf None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:03:21.802832 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d914b37d-966b-4a0f-ac0b-497b49fa12cf None admin] PATH_INFO: `/v3/users/placement` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:03:21.810920 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d914b37d-966b-4a0f-ac0b-497b49fa12cf None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:03:21.812403 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d914b37d-966b-4a0f-ac0b-497b49fa12cf None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:03:21.824063 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-d914b37d-966b-4a0f-ac0b-497b49fa12cf None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 09 12:03:21.824063 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:03:21.824063 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:03:21.824063 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:03:21.824063 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:03:21.824063 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:03:21.824063 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:03:21.824063 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:03:21.824063 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:03:21.824063 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:03:21.824063 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:03:21.824063 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:03:21.824063 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 12:03:21.824063 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 12:03:21.824063 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 12:03:21.824063 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 12:03:21.825330 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:03:21.825330 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:03:21.825330 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 12:03:21.825330 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:03:21.825330 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 12:03:21.825330 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:03:21.825330 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:03:21.825330 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:03:21.825330 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:03:21.825330 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:03:21.825330 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:03:21.825330 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 12:03:21.825330 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:03:21.825330 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:03:21.825330 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:03:21.825330 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:03:21.825330 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:03:21.825330 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:03:21.826602 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:03:21.826602 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:03:21.826602 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 12:03:21.826602 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 12:03:21.826602 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 12:03:21.826602 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 12:03:21.826602 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 12:03:21.826602 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 12:03:21.826602 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 09 12:03:21.826602 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:03:21.826602 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 628/1267] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 12:03:21 2022] GET /identity/v3/users/placement => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:03:21.830965 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8e5aad-1463-408c-a8cf-c5cc90a2cf48 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:03:21.925358 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8e5aad-1463-408c-a8cf-c5cc90a2cf48 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:03:21.958092 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96568c9-358e-4fe6-b4de-566bbc155e00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:03:21.958681 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96568c9-358e-4fe6-b4de-566bbc155e00 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:03:21.958985 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96568c9-358e-4fe6-b4de-566bbc155e00 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:03:21.959256 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96568c9-358e-4fe6-b4de-566bbc155e00 None admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:03:21.959963 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96568c9-358e-4fe6-b4de-566bbc155e00 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:03:21.960268 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96568c9-358e-4fe6-b4de-566bbc155e00 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:03:21.961870 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96568c9-358e-4fe6-b4de-566bbc155e00 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:03:21.970161 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:03:21.979501 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 640/1268] 10.222.0.49 () {56 vars in 1143 bytes} [Fri Sep 9 12:03:21 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:03:21.985190 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d914b37d-966b-4a0f-ac0b-497b49fa12cf None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:03:22.082803 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d914b37d-966b-4a0f-ac0b-497b49fa12cf None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:03:22.119760 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a626ae-04a0-4de5-9494-e8b39ebe2a07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:03:22.120394 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a626ae-04a0-4de5-9494-e8b39ebe2a07 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:03:22.120726 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a626ae-04a0-4de5-9494-e8b39ebe2a07 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:03:22.121010 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a626ae-04a0-4de5-9494-e8b39ebe2a07 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:03:22.131478 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a626ae-04a0-4de5-9494-e8b39ebe2a07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:03:22.132908 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a626ae-04a0-4de5-9494-e8b39ebe2a07 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:03:22.146580 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-e1a626ae-04a0-4de5-9494-e8b39ebe2a07 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:03:22.146580 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:03:22.146580 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:03:22.146580 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:03:22.146580 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:03:22.146580 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:03:22.146580 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:03:22.146580 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:03:22.146580 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:03:22.146580 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:03:22.146580 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:03:22.146580 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:03:22.146580 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:03:22.146580 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:03:22.146580 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:03:22.146580 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:03:22.147634 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:03:22.147634 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:03:22.147634 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:03:22.147634 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:03:22.147634 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:03:22.147634 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:03:22.147634 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:03:22.147634 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 12:03:22.147634 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:03:22.147634 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:03:22.147634 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:03:22.147634 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:03:22.147634 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:03:22.147634 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:03:22.147634 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:03:22.147634 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:03:22.147634 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:03:22.148809 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:03:22.148809 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:03:22.148809 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:03:22.148809 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 629/1269] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 12:03:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:03:22.153485 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96568c9-358e-4fe6-b4de-566bbc155e00 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:03:22.250121 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96568c9-358e-4fe6-b4de-566bbc155e00 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:03:22.282655 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702d1224-5a40-474e-a3f3-2f3f198457aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:03:22.283271 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702d1224-5a40-474e-a3f3-2f3f198457aa None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:03:22.283549 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702d1224-5a40-474e-a3f3-2f3f198457aa None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:03:22.283818 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702d1224-5a40-474e-a3f3-2f3f198457aa None admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:03:22.284445 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702d1224-5a40-474e-a3f3-2f3f198457aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:03:22.284741 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702d1224-5a40-474e-a3f3-2f3f198457aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:03:22.286090 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702d1224-5a40-474e-a3f3-2f3f198457aa None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:03:22.297470 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 641/1270] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 9 12:03:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:03:22.303099 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a626ae-04a0-4de5-9494-e8b39ebe2a07 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:03:22.333735 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:03:22.400679 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a626ae-04a0-4de5-9494-e8b39ebe2a07 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:03:22.433310 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1a5d6d-2536-4b89-bff7-cf5f01982964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:03:22.433923 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1a5d6d-2536-4b89-bff7-cf5f01982964 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:03:22.434200 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1a5d6d-2536-4b89-bff7-cf5f01982964 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:03:22.434470 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1a5d6d-2536-4b89-bff7-cf5f01982964 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:03:22.441106 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1a5d6d-2536-4b89-bff7-cf5f01982964 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:03:22.442539 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1a5d6d-2536-4b89-bff7-cf5f01982964 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:03:22.453402 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-5f1a5d6d-2536-4b89-bff7-cf5f01982964 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:03:22.453402 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:03:22.453402 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:03:22.453402 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:03:22.453402 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:03:22.453402 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:03:22.453402 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:03:22.453402 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:03:22.453402 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:03:22.453402 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:03:22.453402 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:03:22.453402 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:03:22.453402 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 12:03:22.453402 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 12:03:22.453402 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 12:03:22.454457 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 12:03:22.454457 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:03:22.454457 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:03:22.454457 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:03:22.454457 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:03:22.454457 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:03:22.454457 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:03:22.454457 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:03:22.454457 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 12:03:22.454457 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:03:22.454457 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:03:22.454457 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:03:22.454457 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:03:22.454457 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:03:22.454457 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:03:22.454457 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:03:22.454457 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:03:22.455696 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 12:03:22.455696 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 12:03:22.455696 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 12:03:22.455696 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 12:03:22.455696 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 12:03:22.455696 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 12:03:22.455696 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:03:22.455696 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:03:22.455696 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 630/1271] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 9 12:03:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:03:22.460189 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702d1224-5a40-474e-a3f3-2f3f198457aa None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:03:22.517679 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:03:22.558454 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702d1224-5a40-474e-a3f3-2f3f198457aa None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:03:22.591361 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb568e9-07cf-4fac-9708-fb3fe5db5485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:03:22.591953 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb568e9-07cf-4fac-9708-fb3fe5db5485 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:03:22.592231 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb568e9-07cf-4fac-9708-fb3fe5db5485 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:03:22.592504 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb568e9-07cf-4fac-9708-fb3fe5db5485 None admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:03:22.593175 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb568e9-07cf-4fac-9708-fb3fe5db5485 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:03:22.593471 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb568e9-07cf-4fac-9708-fb3fe5db5485 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:03:22.594887 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb568e9-07cf-4fac-9708-fb3fe5db5485 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:03:22.606290 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 642/1272] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Sep 9 12:03:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:03:22.612381 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1a5d6d-2536-4b89-bff7-cf5f01982964 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:03:22.703376 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:03:22.714030 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1a5d6d-2536-4b89-bff7-cf5f01982964 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:03:22.746723 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e4b79d-876d-4639-87a7-5a4ac3d192e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:03:22.747419 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e4b79d-876d-4639-87a7-5a4ac3d192e6 None admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:03:22.747693 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e4b79d-876d-4639-87a7-5a4ac3d192e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:03:22.747960 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e4b79d-876d-4639-87a7-5a4ac3d192e6 None admin] PATH_INFO: `/v3/projects/fe863fcd75f1406ebbffc5016078f73e/users/d3adb2f0583c4ff782049dacdb3ee556/roles/3f2ca33902a04b6dbc2ef7d06d7289d3` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:03:22.749785 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e4b79d-876d-4639-87a7-5a4ac3d192e6 None admin] RBAC: Authorizing `identity:create_grant(project_id=fe863fcd75f1406ebbffc5016078f73e, user_id=d3adb2f0583c4ff782049dacdb3ee556, role_id=3f2ca33902a04b6dbc2ef7d06d7289d3)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:03:22.751232 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e4b79d-876d-4639-87a7-5a4ac3d192e6 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:03:22.764060 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 631/1273] 10.222.0.49 () {58 vars in 1439 bytes} [Fri Sep 9 12:03:22 2022] PUT /identity/v3/projects/fe863fcd75f1406ebbffc5016078f73e/users/d3adb2f0583c4ff782049dacdb3ee556/roles/3f2ca33902a04b6dbc2ef7d06d7289d3 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:03:24.889899 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ee654c-4673-4090-839b-9b3a75bb84e2 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:03:24.890448 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ee654c-4673-4090-839b-9b3a75bb84e2 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:03:24.890448 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ee654c-4673-4090-839b-9b3a75bb84e2 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:03:24.891806 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 643/1274] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:03:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:03:24.897045 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65bf0a0-63a4-4623-a691-2f8396779a1d None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:03:24.897458 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65bf0a0-63a4-4623-a691-2f8396779a1d None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:03:24.897550 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65bf0a0-63a4-4623-a691-2f8396779a1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:03:24.916165 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-f65bf0a0-63a4-4623-a691-2f8396779a1d None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:03:24.917335 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-f65bf0a0-63a4-4623-a691-2f8396779a1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:03:24.981188 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 632/1275] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:03:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 12:03:25.252043 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ee654c-4673-4090-839b-9b3a75bb84e2 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:03:25.350973 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ee654c-4673-4090-839b-9b3a75bb84e2 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:03:25.367904 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:03:25.384617 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e51a8c3-918d-4151-986e-efcd14f230a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:03:25.385234 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e51a8c3-918d-4151-986e-efcd14f230a7 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:03:25.385513 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e51a8c3-918d-4151-986e-efcd14f230a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:03:25.385780 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e51a8c3-918d-4151-986e-efcd14f230a7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:03:25.395552 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e51a8c3-918d-4151-986e-efcd14f230a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 12:03:25.400346 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-5e51a8c3-918d-4151-986e-efcd14f230a7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 12:03:25.400346 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:03:25.400346 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:03:25.400346 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:03:25.400346 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:03:25.400346 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:03:25.400346 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:03:25.400346 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:03:25.400346 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:03:25.400346 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:03:25.400346 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:03:25.400346 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:03:25.400346 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 12:03:25.400346 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 12:03:25.400346 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 12:03:25.400346 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 12:03:25.401435 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 12:03:25.401435 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 12:03:25.401435 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 12:03:25.401435 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 12:03:25.401435 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:03:25.401435 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:03:25.401435 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:03:25.401435 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:03:25.401435 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:03:25.401435 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:03:25.401435 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:03:25.401435 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 12:03:25.401435 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:03:25.401435 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:03:25.401435 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:03:25.401435 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:03:25.402546 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:03:25.402546 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:03:25.402546 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:03:25.402546 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:03:25.402546 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 12:03:25.402546 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 12:03:25.402546 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 12:03:25.402546 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 12:03:25.402546 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 12:03:25.402546 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 12:03:25.402546 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 12:03:25.402546 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:03:25.402546 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 644/1276] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Sep 9 12:03:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:03:25.407451 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65bf0a0-63a4-4623-a691-2f8396779a1d None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:03:25.485689 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:03:25.504275 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65bf0a0-63a4-4623-a691-2f8396779a1d None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:03:25.537068 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24640806-4913-471e-a434-93ee5c5c4a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:03:25.537665 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24640806-4913-471e-a434-93ee5c5c4a98 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:03:25.537945 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24640806-4913-471e-a434-93ee5c5c4a98 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:03:25.538216 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24640806-4913-471e-a434-93ee5c5c4a98 None admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:03:25.538865 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24640806-4913-471e-a434-93ee5c5c4a98 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:03:25.539224 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24640806-4913-471e-a434-93ee5c5c4a98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:03:25.540585 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24640806-4913-471e-a434-93ee5c5c4a98 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:03:25.548377 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 633/1277] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 9 12:03:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:03:25.554181 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e51a8c3-918d-4151-986e-efcd14f230a7 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:03:25.651587 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e51a8c3-918d-4151-986e-efcd14f230a7 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:03:25.684492 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9beff1-bf5b-4f93-81a6-b313432107c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:03:25.685367 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9beff1-bf5b-4f93-81a6-b313432107c7 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:03:25.685646 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9beff1-bf5b-4f93-81a6-b313432107c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:03:25.685917 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9beff1-bf5b-4f93-81a6-b313432107c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:03:25.695733 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9beff1-bf5b-4f93-81a6-b313432107c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:03:25.697103 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9beff1-bf5b-4f93-81a6-b313432107c7 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:03:25.711111 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-6a9beff1-bf5b-4f93-81a6-b313432107c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:03:25.711111 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:03:25.711111 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:03:25.711111 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:03:25.711111 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:03:25.711111 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:03:25.711111 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:03:25.711111 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:03:25.711111 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:03:25.711111 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:03:25.711111 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:03:25.711111 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:03:25.711111 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:03:25.711111 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:03:25.711111 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:03:25.711111 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:03:25.712408 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:03:25.712408 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:03:25.712408 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:03:25.712408 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:03:25.712408 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:03:25.712408 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:03:25.712408 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:03:25.712408 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 12:03:25.712408 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:03:25.712408 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:03:25.712408 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:03:25.712408 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:03:25.712408 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:03:25.712408 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:03:25.712408 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:03:25.712408 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:03:25.712408 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:03:25.713509 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:03:25.713509 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:03:25.713509 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:03:25.713509 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 645/1278] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 12:03:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:03:25.717829 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24640806-4913-471e-a434-93ee5c5c4a98 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:03:25.817809 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24640806-4913-471e-a434-93ee5c5c4a98 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:03:25.850322 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946f5dd7-d54a-4ad7-b331-b5af1ca73b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:03:25.850940 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946f5dd7-d54a-4ad7-b331-b5af1ca73b0c None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:03:25.851225 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946f5dd7-d54a-4ad7-b331-b5af1ca73b0c None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:03:25.851999 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946f5dd7-d54a-4ad7-b331-b5af1ca73b0c None admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:03:25.852628 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946f5dd7-d54a-4ad7-b331-b5af1ca73b0c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:03:25.852929 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946f5dd7-d54a-4ad7-b331-b5af1ca73b0c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:03:25.854301 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946f5dd7-d54a-4ad7-b331-b5af1ca73b0c None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:03:25.865612 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 634/1279] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 9 12:03:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:03:25.872124 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9beff1-bf5b-4f93-81a6-b313432107c7 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:03:25.902772 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:03:25.969850 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9beff1-bf5b-4f93-81a6-b313432107c7 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:03:26.002049 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3768262b-4dae-48cf-95c1-7420a688804e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:03:26.002658 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3768262b-4dae-48cf-95c1-7420a688804e None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:03:26.002957 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3768262b-4dae-48cf-95c1-7420a688804e None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:03:26.003232 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3768262b-4dae-48cf-95c1-7420a688804e None admin] PATH_INFO: `/v3/users/placement` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:03:26.011107 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3768262b-4dae-48cf-95c1-7420a688804e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:03:26.012529 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3768262b-4dae-48cf-95c1-7420a688804e None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:03:26.023229 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-3768262b-4dae-48cf-95c1-7420a688804e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 09 12:03:26.023229 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:03:26.023229 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:03:26.023229 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:03:26.023229 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:03:26.023229 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:03:26.023229 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:03:26.023229 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:03:26.023229 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:03:26.023229 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:03:26.023229 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:03:26.023229 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:03:26.023229 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 12:03:26.023229 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 12:03:26.023229 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 12:03:26.023229 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 12:03:26.024693 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:03:26.024693 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:03:26.024693 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 12:03:26.024693 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:03:26.024693 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 12:03:26.024693 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 12:03:26.024693 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:03:26.024693 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:03:26.024693 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:03:26.024693 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:03:26.024693 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:03:26.024693 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 12:03:26.024693 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:03:26.024693 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:03:26.024693 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:03:26.024693 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:03:26.024693 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:03:26.024693 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:03:26.025849 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:03:26.025849 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:03:26.025849 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 12:03:26.025849 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 12:03:26.025849 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 12:03:26.025849 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 12:03:26.025849 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 12:03:26.025849 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 12:03:26.025849 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 09 12:03:26.025849 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:03:26.025849 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 646/1280] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 12:03:25 2022] GET /identity/v3/users/placement => generated 87 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:03:26.030398 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946f5dd7-d54a-4ad7-b331-b5af1ca73b0c None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:03:26.034173 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:03:26.129250 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946f5dd7-d54a-4ad7-b331-b5af1ca73b0c None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:03:26.162419 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0103ce25-bd4a-430c-b784-d5470dd80acd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:03:26.163053 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0103ce25-bd4a-430c-b784-d5470dd80acd None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:03:26.163338 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0103ce25-bd4a-430c-b784-d5470dd80acd None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:03:26.163610 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0103ce25-bd4a-430c-b784-d5470dd80acd None admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:03:26.164366 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0103ce25-bd4a-430c-b784-d5470dd80acd None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:03:26.164688 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0103ce25-bd4a-430c-b784-d5470dd80acd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:03:26.166133 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0103ce25-bd4a-430c-b784-d5470dd80acd None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:03:26.183286 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 635/1281] 10.222.0.49 () {56 vars in 1143 bytes} [Fri Sep 9 12:03:26 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:03:26.189465 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3768262b-4dae-48cf-95c1-7420a688804e None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:03:26.277010 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:03:26.286625 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3768262b-4dae-48cf-95c1-7420a688804e None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:03:26.319406 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9922dd-2e3c-4a2d-85b6-7f89ebe50974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:03:26.319993 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9922dd-2e3c-4a2d-85b6-7f89ebe50974 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:03:26.320878 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9922dd-2e3c-4a2d-85b6-7f89ebe50974 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:03:26.321151 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9922dd-2e3c-4a2d-85b6-7f89ebe50974 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:03:26.384856 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9922dd-2e3c-4a2d-85b6-7f89ebe50974 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:03:26.386230 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9922dd-2e3c-4a2d-85b6-7f89ebe50974 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:03:26.469641 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:03:26.480349 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 647/1282] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 9 12:03:26 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 12:03:26.486210 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0103ce25-bd4a-430c-b784-d5470dd80acd None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:03:26.547971 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:03:26.585698 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0103ce25-bd4a-430c-b784-d5470dd80acd None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:03:26.619263 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ea9f98-bfb3-47bd-a634-fc10122fa04e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:03:26.619882 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ea9f98-bfb3-47bd-a634-fc10122fa04e None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:03:26.620157 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ea9f98-bfb3-47bd-a634-fc10122fa04e None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:03:26.620424 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ea9f98-bfb3-47bd-a634-fc10122fa04e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:03:26.630137 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ea9f98-bfb3-47bd-a634-fc10122fa04e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:03:26.631655 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ea9f98-bfb3-47bd-a634-fc10122fa04e None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:03:26.646012 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-06ea9f98-bfb3-47bd-a634-fc10122fa04e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:03:26.646012 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:03:26.646012 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:03:26.646012 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:03:26.646012 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:03:26.646012 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:03:26.646012 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:03:26.646012 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:03:26.646012 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:03:26.646012 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:03:26.646012 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:03:26.646012 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:03:26.646012 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 12:03:26.646012 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 12:03:26.646012 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 12:03:26.646012 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 12:03:26.647206 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:03:26.647206 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:03:26.647206 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:03:26.647206 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:03:26.647206 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:03:26.647206 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:03:26.647206 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:03:26.647206 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 12:03:26.647206 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:03:26.647206 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:03:26.647206 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:03:26.647206 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:03:26.647206 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:03:26.647206 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:03:26.647206 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:03:26.647206 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:03:26.647206 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 12:03:26.648397 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 12:03:26.648397 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 12:03:26.648397 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:03:26.648397 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 636/1283] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 12:03:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:03:26.653085 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9922dd-2e3c-4a2d-85b6-7f89ebe50974 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:03:26.748615 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9922dd-2e3c-4a2d-85b6-7f89ebe50974 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:03:26.781121 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06f7217-3027-444e-bcb9-3e9dbdac4498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:03:26.781713 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06f7217-3027-444e-bcb9-3e9dbdac4498 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:03:26.781989 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06f7217-3027-444e-bcb9-3e9dbdac4498 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:03:26.782259 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06f7217-3027-444e-bcb9-3e9dbdac4498 None admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:03:26.782884 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06f7217-3027-444e-bcb9-3e9dbdac4498 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:03:26.783211 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06f7217-3027-444e-bcb9-3e9dbdac4498 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:03:26.784551 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06f7217-3027-444e-bcb9-3e9dbdac4498 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:03:26.795956 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 648/1284] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 9 12:03:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:03:26.801624 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ea9f98-bfb3-47bd-a634-fc10122fa04e None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:03:26.899722 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ea9f98-bfb3-47bd-a634-fc10122fa04e None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:03:26.912519 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:03:26.933178 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1db03e-ece2-4258-afa3-004d88455830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:03:26.933789 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1db03e-ece2-4258-afa3-004d88455830 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:03:26.934062 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1db03e-ece2-4258-afa3-004d88455830 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:03:26.934333 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1db03e-ece2-4258-afa3-004d88455830 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:03:26.940942 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1db03e-ece2-4258-afa3-004d88455830 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:03:26.942360 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1db03e-ece2-4258-afa3-004d88455830 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:03:26.953426 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application [None req-da1db03e-ece2-4258-afa3-004d88455830 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:03:26.953426 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:03:26.953426 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:03:26.953426 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:03:26.953426 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:03:26.953426 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:03:26.953426 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:03:26.953426 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:03:26.953426 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:03:26.953426 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:03:26.953426 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:03:26.953426 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:03:26.953426 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 12:03:26.953426 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 12:03:26.953426 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 12:03:26.954471 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 12:03:26.954471 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:03:26.954471 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:03:26.954471 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:03:26.954471 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:03:26.954471 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:03:26.954471 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:03:26.954471 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:03:26.954471 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application with Lock( Sep 09 12:03:26.954471 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:03:26.954471 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:03:26.954471 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:03:26.954471 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:03:26.954471 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:03:26.954471 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:03:26.954471 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:03:26.954471 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:03:26.955702 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 12:03:26.955702 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 12:03:26.955702 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 12:03:26.955702 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 12:03:26.955702 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 12:03:26.955702 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 12:03:26.955702 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 12:03:26.955702 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: ERROR keystone.server.flask.application Sep 09 12:03:26.955702 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 637/1285] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 9 12:03:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:03:26.960530 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06f7217-3027-444e-bcb9-3e9dbdac4498 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:03:27.008935 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:03:27.057141 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06f7217-3027-444e-bcb9-3e9dbdac4498 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:03:27.090169 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813a8112-b698-4344-bf84-99ff54cdf68d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:03:27.090869 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813a8112-b698-4344-bf84-99ff54cdf68d None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:03:27.091264 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813a8112-b698-4344-bf84-99ff54cdf68d None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:03:27.091595 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813a8112-b698-4344-bf84-99ff54cdf68d None admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:03:27.092434 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813a8112-b698-4344-bf84-99ff54cdf68d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:03:27.092893 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813a8112-b698-4344-bf84-99ff54cdf68d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:03:27.094578 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813a8112-b698-4344-bf84-99ff54cdf68d None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:03:27.106213 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 649/1286] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Sep 9 12:03:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:03:27.112236 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1db03e-ece2-4258-afa3-004d88455830 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:03:27.208230 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1db03e-ece2-4258-afa3-004d88455830 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:03:27.240629 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8d9b6a-4b0e-435b-997c-826b2f4a30a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:03:27.241211 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8d9b6a-4b0e-435b-997c-826b2f4a30a8 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:03:27.241483 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8d9b6a-4b0e-435b-997c-826b2f4a30a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:03:27.241748 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8d9b6a-4b0e-435b-997c-826b2f4a30a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:03:27.243116 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8d9b6a-4b0e-435b-997c-826b2f4a30a8 None admin] RBAC: Adding query filter params (role.id=3f2ca33902a04b6dbc2ef7d06d7289d3, scope.project.id=fe863fcd75f1406ebbffc5016078f73e, user.id=d3adb2f0583c4ff782049dacdb3ee556) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:03:27.243417 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8d9b6a-4b0e-435b-997c-826b2f4a30a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:03:27.244823 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8d9b6a-4b0e-435b-997c-826b2f4a30a8 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:03:27.250369 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 638/1287] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Sep 9 12:03:27 2022] GET /identity/v3/role_assignments?user.id=d3adb2f0583c4ff782049dacdb3ee556&scope.project.id=fe863fcd75f1406ebbffc5016078f73e&role.id=3f2ca33902a04b6dbc2ef7d06d7289d3 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:03:29.391353 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4c5f23-7753-4192-b4a4-8405b5d12815 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:03:29.391883 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4c5f23-7753-4192-b4a4-8405b5d12815 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:03:29.391883 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4c5f23-7753-4192-b4a4-8405b5d12815 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:03:29.393225 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 650/1288] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:03:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:03:29.398459 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011c3e14-950e-49b8-a306-1e560e657e37 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:03:29.398993 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011c3e14-950e-49b8-a306-1e560e657e37 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:03:29.399080 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011c3e14-950e-49b8-a306-1e560e657e37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:03:29.417374 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-011c3e14-950e-49b8-a306-1e560e657e37 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:03:29.418575 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-011c3e14-950e-49b8-a306-1e560e657e37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:03:29.447997 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:03:29.484417 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 639/1289] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:03:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 12:03:29.749582 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4c5f23-7753-4192-b4a4-8405b5d12815 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:03:29.847898 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4c5f23-7753-4192-b4a4-8405b5d12815 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:03:29.858722 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:03:29.881241 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1512276-7863-412c-9ecd-0d135c402048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:03:29.881858 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1512276-7863-412c-9ecd-0d135c402048 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:03:29.882135 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1512276-7863-412c-9ecd-0d135c402048 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:03:29.882404 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1512276-7863-412c-9ecd-0d135c402048 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:03:29.883154 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1512276-7863-412c-9ecd-0d135c402048 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:03:29.884505 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1512276-7863-412c-9ecd-0d135c402048 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:03:29.892981 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-e1512276-7863-412c-9ecd-0d135c402048 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 12:03:29.892981 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:03:29.892981 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 12:03:29.892981 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 12:03:29.892981 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 12:03:29.892981 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 12:03:29.892981 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 12:03:29.892981 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 12:03:29.892981 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 12:03:29.892981 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:03:29.892981 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 12:03:29.892981 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:03:29.892981 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:03:29.892981 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:03:29.892981 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:03:29.892981 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:03:29.892981 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:03:29.894226 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:03:29.894226 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:03:29.894226 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:03:29.894226 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:03:29.894226 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:03:29.894226 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:03:29.894226 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 12:03:29.894226 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 12:03:29.894226 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 12:03:29.894226 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 12:03:29.894226 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:03:29.894226 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:03:29.894226 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:03:29.894226 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:03:29.894226 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:03:29.894226 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:03:29.895177 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:03:29.895177 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 12:03:29.895177 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:03:29.895177 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:03:29.895177 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:03:29.895177 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:03:29.895177 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:03:29.895177 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:03:29.895177 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:03:29.895177 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:03:29.895177 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 12:03:29.895177 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 12:03:29.895177 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 12:03:29.895177 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:03:29.895177 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 651/1290] 10.222.0.49 () {56 vars in 1117 bytes} [Fri Sep 9 12:03:29 2022] GET /identity/v3/services/placement => generated 90 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:03:29.900697 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011c3e14-950e-49b8-a306-1e560e657e37 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:03:29.996882 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011c3e14-950e-49b8-a306-1e560e657e37 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:03:30.029515 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352e96c7-eea5-479b-8b35-bde1c70f1850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:03:30.030103 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352e96c7-eea5-479b-8b35-bde1c70f1850 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:03:30.030378 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352e96c7-eea5-479b-8b35-bde1c70f1850 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:03:30.030644 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352e96c7-eea5-479b-8b35-bde1c70f1850 None admin] PATH_INFO: `/v3/services` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:03:30.031298 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352e96c7-eea5-479b-8b35-bde1c70f1850 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:03:30.031595 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352e96c7-eea5-479b-8b35-bde1c70f1850 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:03:30.032954 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352e96c7-eea5-479b-8b35-bde1c70f1850 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:03:30.037559 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 640/1291] 10.222.0.49 () {56 vars in 1116 bytes} [Fri Sep 9 12:03:29 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:03:30.043242 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1512276-7863-412c-9ecd-0d135c402048 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:03:30.147821 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1512276-7863-412c-9ecd-0d135c402048 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:03:30.182661 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5ff15c-91c3-4ab3-8317-1045300581a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:03:30.183289 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5ff15c-91c3-4ab3-8317-1045300581a2 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:03:30.183567 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5ff15c-91c3-4ab3-8317-1045300581a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:03:30.183837 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5ff15c-91c3-4ab3-8317-1045300581a2 None admin] PATH_INFO: `/v3/services` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:03:30.184456 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5ff15c-91c3-4ab3-8317-1045300581a2 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:03:30.184755 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5ff15c-91c3-4ab3-8317-1045300581a2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:03:30.186353 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5ff15c-91c3-4ab3-8317-1045300581a2 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:03:30.190663 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 652/1292] 10.222.0.49 () {56 vars in 1116 bytes} [Fri Sep 9 12:03:30 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:03:32.335348 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c44319-91d3-433a-b9e8-c61ec867ae2d None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:03:32.336058 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c44319-91d3-433a-b9e8-c61ec867ae2d None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:03:32.336058 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c44319-91d3-433a-b9e8-c61ec867ae2d None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:03:32.337384 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 641/1293] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:03:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:03:32.342453 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e9e473-7ce6-439c-8ccc-87979e01dfbb None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:03:32.342826 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e9e473-7ce6-439c-8ccc-87979e01dfbb None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:03:32.343027 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e9e473-7ce6-439c-8ccc-87979e01dfbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:03:32.361117 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-b5e9e473-7ce6-439c-8ccc-87979e01dfbb None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:03:32.362288 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-b5e9e473-7ce6-439c-8ccc-87979e01dfbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:03:32.390654 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:03:32.426858 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 653/1294] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:03:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 12:03:32.704153 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c44319-91d3-433a-b9e8-c61ec867ae2d None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:03:32.734318 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:03:32.804025 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c44319-91d3-433a-b9e8-c61ec867ae2d None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:03:32.838013 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701bf43c-740a-437c-953e-f2c38b2f21d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:03:32.838597 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701bf43c-740a-437c-953e-f2c38b2f21d5 None admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:03:32.838867 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701bf43c-740a-437c-953e-f2c38b2f21d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:03:32.839162 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701bf43c-740a-437c-953e-f2c38b2f21d5 None admin] PATH_INFO: `/v3/services` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:03:32.839808 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701bf43c-740a-437c-953e-f2c38b2f21d5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:03:32.841151 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701bf43c-740a-437c-953e-f2c38b2f21d5 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:03:32.854867 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 642/1295] 10.222.0.49 () {60 vars in 1141 bytes} [Fri Sep 9 12:03:32 2022] POST /identity/v3/services => generated 246 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:03:35.008009 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779d96a3-8e70-421d-bbf2-6feabae15b46 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:03:35.008523 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779d96a3-8e70-421d-bbf2-6feabae15b46 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:03:35.008523 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779d96a3-8e70-421d-bbf2-6feabae15b46 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:03:35.009888 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 654/1296] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:03:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:03:35.015080 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b56cf9-3fac-4ad9-abf5-17ec746f9eb2 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:03:35.015502 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b56cf9-3fac-4ad9-abf5-17ec746f9eb2 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:03:35.015605 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b56cf9-3fac-4ad9-abf5-17ec746f9eb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:03:35.033124 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-b0b56cf9-3fac-4ad9-abf5-17ec746f9eb2 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:03:35.034302 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-b0b56cf9-3fac-4ad9-abf5-17ec746f9eb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:03:35.107155 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 643/1297] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:03:35 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 12:03:35.389303 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779d96a3-8e70-421d-bbf2-6feabae15b46 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:03:35.486810 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779d96a3-8e70-421d-bbf2-6feabae15b46 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:03:35.519877 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561ab4b8-f4cb-47d6-980e-5e199a083e20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:03:35.520492 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561ab4b8-f4cb-47d6-980e-5e199a083e20 None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:03:35.520770 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561ab4b8-f4cb-47d6-980e-5e199a083e20 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:03:35.521040 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561ab4b8-f4cb-47d6-980e-5e199a083e20 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:03:35.521616 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561ab4b8-f4cb-47d6-980e-5e199a083e20 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:03:35.523622 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561ab4b8-f4cb-47d6-980e-5e199a083e20 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:03:35.532177 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-561ab4b8-f4cb-47d6-980e-5e199a083e20 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 12:03:35.532177 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:03:35.532177 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 12:03:35.532177 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 12:03:35.532177 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 12:03:35.532177 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 12:03:35.532177 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 12:03:35.532177 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 12:03:35.532177 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 12:03:35.532177 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:03:35.532177 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 12:03:35.532177 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:03:35.532177 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:03:35.532177 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:03:35.532177 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:03:35.532177 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:03:35.532177 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:03:35.533351 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:03:35.533351 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:03:35.533351 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:03:35.533351 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:03:35.533351 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:03:35.533351 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:03:35.533351 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 12:03:35.533351 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 12:03:35.533351 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 12:03:35.533351 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 12:03:35.533351 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:03:35.533351 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:03:35.533351 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:03:35.533351 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:03:35.533351 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:03:35.533351 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:03:35.534594 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:03:35.534594 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 12:03:35.534594 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:03:35.534594 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:03:35.534594 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:03:35.534594 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:03:35.534594 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:03:35.534594 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:03:35.534594 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:03:35.534594 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:03:35.534594 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 12:03:35.534594 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 12:03:35.534594 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 12:03:35.534594 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:03:35.534594 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 655/1298] 10.222.0.49 () {56 vars in 1117 bytes} [Fri Sep 9 12:03:35 2022] GET /identity/v3/services/placement => generated 90 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:03:35.539943 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b56cf9-3fac-4ad9-abf5-17ec746f9eb2 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:03:35.558155 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:03:35.637117 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b56cf9-3fac-4ad9-abf5-17ec746f9eb2 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:03:35.670515 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910e1c39-3bdb-4559-93ee-4fde54248cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:03:35.671143 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910e1c39-3bdb-4559-93ee-4fde54248cd4 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:03:35.671419 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910e1c39-3bdb-4559-93ee-4fde54248cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:03:35.671684 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910e1c39-3bdb-4559-93ee-4fde54248cd4 None admin] PATH_INFO: `/v3/services` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:03:35.672299 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910e1c39-3bdb-4559-93ee-4fde54248cd4 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:03:35.672597 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910e1c39-3bdb-4559-93ee-4fde54248cd4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:03:35.673955 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910e1c39-3bdb-4559-93ee-4fde54248cd4 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:03:35.679665 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 644/1299] 10.222.0.49 () {56 vars in 1116 bytes} [Fri Sep 9 12:03:35 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:03:35.685909 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561ab4b8-f4cb-47d6-980e-5e199a083e20 None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:03:35.716334 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:03:35.782119 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561ab4b8-f4cb-47d6-980e-5e199a083e20 None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:03:35.814560 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dca0d67-842f-4783-85e1-6cb2a1f8d8fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:03:35.815176 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dca0d67-842f-4783-85e1-6cb2a1f8d8fd None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:03:35.815452 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dca0d67-842f-4783-85e1-6cb2a1f8d8fd None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:03:35.815719 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dca0d67-842f-4783-85e1-6cb2a1f8d8fd None admin] PATH_INFO: `/v3/endpoints` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:03:35.816403 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dca0d67-842f-4783-85e1-6cb2a1f8d8fd None admin] RBAC: Adding query filter params (interface=public, service_id=af2c4f542b1a40ea9095687032fd91b8, region_id=RegionOne) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:03:35.816708 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dca0d67-842f-4783-85e1-6cb2a1f8d8fd None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:03:35.818049 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dca0d67-842f-4783-85e1-6cb2a1f8d8fd None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:03:35.822881 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 656/1300] 10.222.0.49 () {56 vars in 1251 bytes} [Fri Sep 9 12:03:35 2022] GET /identity/v3/endpoints?service_id=af2c4f542b1a40ea9095687032fd91b8&interface=public®ion_id=RegionOne => generated 194 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:03:35.828851 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910e1c39-3bdb-4559-93ee-4fde54248cd4 None admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:03:35.908674 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:03:35.927637 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910e1c39-3bdb-4559-93ee-4fde54248cd4 None admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:03:35.960631 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b5a31b-c050-4af5-a65f-01942364044c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:03:35.961333 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b5a31b-c050-4af5-a65f-01942364044c None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:03:35.961611 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b5a31b-c050-4af5-a65f-01942364044c None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:03:35.961879 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b5a31b-c050-4af5-a65f-01942364044c None admin] PATH_INFO: `/v3/services` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:03:35.962473 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b5a31b-c050-4af5-a65f-01942364044c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:03:35.963847 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b5a31b-c050-4af5-a65f-01942364044c None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:03:35.970424 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 645/1301] 10.222.0.49 () {56 vars in 1087 bytes} [Fri Sep 9 12:03:35 2022] GET /identity/v3/services => generated 2191 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:03:38.086542 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7794ac2-6914-4e71-a5e3-aff4e79dc522 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:03:38.087120 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7794ac2-6914-4e71-a5e3-aff4e79dc522 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:03:38.087120 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7794ac2-6914-4e71-a5e3-aff4e79dc522 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:03:38.088443 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 657/1302] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:03:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:03:38.094850 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638e7a8c-ff66-4c76-819c-2f83134223af None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:03:38.095411 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638e7a8c-ff66-4c76-819c-2f83134223af None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:03:38.095526 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638e7a8c-ff66-4c76-819c-2f83134223af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:03:38.116389 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-638e7a8c-ff66-4c76-819c-2f83134223af None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:03:38.117672 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-638e7a8c-ff66-4c76-819c-2f83134223af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:03:38.184972 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 646/1303] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:03:38 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 12:03:38.456625 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7794ac2-6914-4e71-a5e3-aff4e79dc522 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:03:38.548817 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:03:38.555642 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7794ac2-6914-4e71-a5e3-aff4e79dc522 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:03:38.588303 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9994898-7e88-48f6-bdd9-337a7816430b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:03:38.588914 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9994898-7e88-48f6-bdd9-337a7816430b None admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:03:38.589191 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9994898-7e88-48f6-bdd9-337a7816430b None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:03:38.589459 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9994898-7e88-48f6-bdd9-337a7816430b None admin] PATH_INFO: `/v3/services/placement` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:03:38.590034 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9994898-7e88-48f6-bdd9-337a7816430b None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:03:38.591422 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9994898-7e88-48f6-bdd9-337a7816430b None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:03:38.599857 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application [None req-c9994898-7e88-48f6-bdd9-337a7816430b None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 12:03:38.599857 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:03:38.599857 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 12:03:38.599857 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 12:03:38.599857 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 12:03:38.599857 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 12:03:38.599857 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 12:03:38.599857 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 12:03:38.599857 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 12:03:38.599857 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:03:38.599857 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 12:03:38.599857 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:03:38.599857 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 12:03:38.599857 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 12:03:38.599857 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 12:03:38.599857 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 12:03:38.599857 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 12:03:38.601023 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 12:03:38.601023 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 12:03:38.601023 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 12:03:38.601023 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 12:03:38.601023 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 12:03:38.601023 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 12:03:38.601023 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 12:03:38.601023 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 12:03:38.601023 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 12:03:38.601023 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 12:03:38.601023 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 12:03:38.601023 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 12:03:38.601023 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 12:03:38.601023 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 12:03:38.601023 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 12:03:38.601023 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 12:03:38.602091 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 12:03:38.602091 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application with Lock( Sep 09 12:03:38.602091 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 12:03:38.602091 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self._enter() Sep 09 12:03:38.602091 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 12:03:38.602091 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 12:03:38.602091 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 12:03:38.602091 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application return self.creator() Sep 09 12:03:38.602091 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 12:03:38.602091 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application created_value = creator( Sep 09 12:03:38.602091 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 12:03:38.602091 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 12:03:38.602091 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 12:03:38.602091 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: ERROR keystone.server.flask.application Sep 09 12:03:38.602091 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 658/1304] 10.222.0.49 () {56 vars in 1117 bytes} [Fri Sep 9 12:03:38 2022] GET /identity/v3/services/placement => generated 90 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 12:03:38.607207 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638e7a8c-ff66-4c76-819c-2f83134223af None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:03:38.682419 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:03:38.705750 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638e7a8c-ff66-4c76-819c-2f83134223af None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:03:38.738381 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91093835-ed24-4855-8e5c-7ed8b423b567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:03:38.738925 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91093835-ed24-4855-8e5c-7ed8b423b567 None admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:03:38.739211 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91093835-ed24-4855-8e5c-7ed8b423b567 None admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:03:38.739474 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91093835-ed24-4855-8e5c-7ed8b423b567 None admin] PATH_INFO: `/v3/services` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:03:38.740085 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91093835-ed24-4855-8e5c-7ed8b423b567 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:03:38.740380 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91093835-ed24-4855-8e5c-7ed8b423b567 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:03:38.742646 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91093835-ed24-4855-8e5c-7ed8b423b567 None admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:03:38.747632 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 647/1305] 10.222.0.49 () {56 vars in 1116 bytes} [Fri Sep 9 12:03:38 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:03:38.753397 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9994898-7e88-48f6-bdd9-337a7816430b None admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:03:38.851274 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9994898-7e88-48f6-bdd9-337a7816430b None admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:03:38.887598 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4942b9-6533-40ca-bad0-28c6fae4e2d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:03:38.888193 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4942b9-6533-40ca-bad0-28c6fae4e2d8 None admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:03:38.888467 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4942b9-6533-40ca-bad0-28c6fae4e2d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:03:38.888732 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4942b9-6533-40ca-bad0-28c6fae4e2d8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:03:38.889391 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4942b9-6533-40ca-bad0-28c6fae4e2d8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:03:38.890738 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4942b9-6533-40ca-bad0-28c6fae4e2d8 None admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:03:38.909072 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 659/1306] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:03:38 2022] POST /identity/v3/endpoints => generated 331 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:03:48.744728 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=30 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:03:48.905094 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:04:43.066846 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d19b7d8-e1c1-4934-ac45-a487acb22aee None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:04:43.067355 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d19b7d8-e1c1-4934-ac45-a487acb22aee None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:04:43.067436 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d19b7d8-e1c1-4934-ac45-a487acb22aee None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:04:43.072648 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 648/1307] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:04:43 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:04:43.074056 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c905a0b-6f3a-4993-bee7-e3b011000864 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:04:43.074502 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c905a0b-6f3a-4993-bee7-e3b011000864 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:04:43.074748 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c905a0b-6f3a-4993-bee7-e3b011000864 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:04:43.194676 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-2c905a0b-6f3a-4993-bee7-e3b011000864 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:04:43.196204 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-2c905a0b-6f3a-4993-bee7-e3b011000864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:04:43.327654 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 660/1308] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:04:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:04:43.618747 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958c3284-0489-41b4-8008-c9b079d69636 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:04:43.619412 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958c3284-0489-41b4-8008-c9b079d69636 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:04:43.619758 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958c3284-0489-41b4-8008-c9b079d69636 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:04:43.621363 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 649/1309] 10.222.0.49 () {54 vars in 965 bytes} [Fri Sep 9 12:04:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:04:43.630307 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8a399e-0aae-4cf0-a3ef-17110587ed14 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:04:43.630612 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8a399e-0aae-4cf0-a3ef-17110587ed14 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:04:43.631196 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8a399e-0aae-4cf0-a3ef-17110587ed14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:04:43.719712 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-4e8a399e-0aae-4cf0-a3ef-17110587ed14 None None] MFA Rules not processed for user `992de6f86b42495dafe73cb2c393fb4e`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:04:43.721271 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-4e8a399e-0aae-4cf0-a3ef-17110587ed14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:04:43.743703 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:04:43.805601 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 661/1310] 10.222.0.49 () {58 vars in 1062 bytes} [Fri Sep 9 12:04:43 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:04:43.815090 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958c3284-0489-41b4-8008-c9b079d69636 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:04:43.816858 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-958c3284-0489-41b4-8008-c9b079d69636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:04:43.933107 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958c3284-0489-41b4-8008-c9b079d69636 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:04:44.012490 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53ec611-3057-424b-9df7-d2399d33b80e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:04:44.013099 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53ec611-3057-424b-9df7-d2399d33b80e service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:04:44.013375 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53ec611-3057-424b-9df7-d2399d33b80e service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:04:44.013641 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53ec611-3057-424b-9df7-d2399d33b80e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:04:44.015263 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-f53ec611-3057-424b-9df7-d2399d33b80e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:04:44.074187 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:04:44.136324 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53ec611-3057-424b-9df7-d2399d33b80e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:04:44.137758 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53ec611-3057-424b-9df7-d2399d33b80e service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:04:44.156942 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 650/1311] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:04:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:04:46.657517 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd9c857-0a12-41af-8afe-6ba83de05262 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:04:46.658586 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd9c857-0a12-41af-8afe-6ba83de05262 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:04:46.658713 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd9c857-0a12-41af-8afe-6ba83de05262 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:04:46.660048 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 662/1312] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:04:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:04:46.665563 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a54104-4dd0-4e9e-8ece-8ea159515859 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:04:46.665998 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a54104-4dd0-4e9e-8ece-8ea159515859 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:04:46.666116 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a54104-4dd0-4e9e-8ece-8ea159515859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:04:46.738542 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-e7a54104-4dd0-4e9e-8ece-8ea159515859 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:04:46.739793 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-e7a54104-4dd0-4e9e-8ece-8ea159515859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:04:46.837179 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 651/1313] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:04:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:04:47.122593 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd9c857-0a12-41af-8afe-6ba83de05262 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:04:47.124169 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-8fd9c857-0a12-41af-8afe-6ba83de05262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:04:47.231666 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd9c857-0a12-41af-8afe-6ba83de05262 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:04:47.304451 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:04:47.306555 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bd3c28-5f04-43a7-b673-ce813c3592bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:04:47.307406 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bd3c28-5f04-43a7-b673-ce813c3592bd service neutron] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:04:47.307873 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bd3c28-5f04-43a7-b673-ce813c3592bd service neutron] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:04:47.308311 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bd3c28-5f04-43a7-b673-ce813c3592bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:04:47.310077 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-a9bd3c28-5f04-43a7-b673-ce813c3592bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:04:47.417549 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bd3c28-5f04-43a7-b673-ce813c3592bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:04:47.418928 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bd3c28-5f04-43a7-b673-ce813c3592bd service neutron] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:04:47.437124 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 663/1314] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:04:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:04:49.879554 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8e62cf-a933-4ef2-b58a-78e58ea90bb1 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:04:49.880077 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8e62cf-a933-4ef2-b58a-78e58ea90bb1 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:04:49.880077 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8e62cf-a933-4ef2-b58a-78e58ea90bb1 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:04:49.881502 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 652/1315] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:04:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:04:49.886755 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9d84f9-a392-4c02-904a-cd77c6316a6c None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:04:49.887213 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9d84f9-a392-4c02-904a-cd77c6316a6c None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:04:49.887328 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9d84f9-a392-4c02-904a-cd77c6316a6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:04:49.951790 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-aa9d84f9-a392-4c02-904a-cd77c6316a6c None None] MFA Rules not processed for user `1ccfba61dc7d4ad98da496e5800a8589`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:04:49.952896 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-aa9d84f9-a392-4c02-904a-cd77c6316a6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:04:50.004349 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:04:50.047493 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 664/1316] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:04:49 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:04:50.335671 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8e62cf-a933-4ef2-b58a-78e58ea90bb1 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:04:50.337277 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-9e8e62cf-a933-4ef2-b58a-78e58ea90bb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:04:50.347911 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:04:50.442307 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8e62cf-a933-4ef2-b58a-78e58ea90bb1 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:04:50.513776 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eae17c-1989-4dbf-85b1-0cbfc1ecb488 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:04:50.514370 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eae17c-1989-4dbf-85b1-0cbfc1ecb488 service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:04:50.514644 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eae17c-1989-4dbf-85b1-0cbfc1ecb488 service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:04:50.514931 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eae17c-1989-4dbf-85b1-0cbfc1ecb488 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:04:50.516509 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-29eae17c-1989-4dbf-85b1-0cbfc1ecb488 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:04:50.577340 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:04:50.624666 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29eae17c-1989-4dbf-85b1-0cbfc1ecb488 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:04:50.626003 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29eae17c-1989-4dbf-85b1-0cbfc1ecb488 service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:04:50.645839 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 653/1317] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:04:50 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:04:53.334380 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b06d3cb-3a51-40b6-ba8f-0b3e66a09f99 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:04:53.334939 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b06d3cb-3a51-40b6-ba8f-0b3e66a09f99 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:04:53.334939 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b06d3cb-3a51-40b6-ba8f-0b3e66a09f99 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:04:53.336304 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 665/1318] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:04:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:04:53.341613 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebd09fc-009d-4670-9254-102012587d6a None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:04:53.342042 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebd09fc-009d-4670-9254-102012587d6a None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:04:53.342198 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebd09fc-009d-4670-9254-102012587d6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:04:53.406641 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-1ebd09fc-009d-4670-9254-102012587d6a None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:04:53.408428 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-1ebd09fc-009d-4670-9254-102012587d6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:04:53.505266 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 654/1319] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:04:53 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:04:53.797323 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b06d3cb-3a51-40b6-ba8f-0b3e66a09f99 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:04:53.799774 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-6b06d3cb-3a51-40b6-ba8f-0b3e66a09f99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:04:53.901814 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b06d3cb-3a51-40b6-ba8f-0b3e66a09f99 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:04:53.972743 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be5f7ce-519b-4831-85af-3a480bad8d2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:04:53.973340 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be5f7ce-519b-4831-85af-3a480bad8d2b service neutron] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:04:53.973612 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be5f7ce-519b-4831-85af-3a480bad8d2b service neutron] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:04:53.973876 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be5f7ce-519b-4831-85af-3a480bad8d2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:04:53.975488 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-5be5f7ce-519b-4831-85af-3a480bad8d2b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:04:53.982112 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:04:54.082306 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be5f7ce-519b-4831-85af-3a480bad8d2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:04:54.083912 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be5f7ce-519b-4831-85af-3a480bad8d2b service neutron] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:04:54.103167 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 666/1320] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:04:53 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:04:56.925419 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356bd6f8-fc41-463d-98e9-8e07b5a5ff4c None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:04:56.925943 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356bd6f8-fc41-463d-98e9-8e07b5a5ff4c None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:04:56.925943 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356bd6f8-fc41-463d-98e9-8e07b5a5ff4c None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:04:56.927258 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 655/1321] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:04:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:04:56.932707 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217d1ca4-ef74-4665-8491-0621a81924a1 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:04:56.933147 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217d1ca4-ef74-4665-8491-0621a81924a1 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:04:56.933255 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217d1ca4-ef74-4665-8491-0621a81924a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:04:57.008876 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-217d1ca4-ef74-4665-8491-0621a81924a1 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:04:57.010102 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-217d1ca4-ef74-4665-8491-0621a81924a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:04:57.074583 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:04:57.118743 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 667/1322] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:04:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:04:57.410100 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356bd6f8-fc41-463d-98e9-8e07b5a5ff4c None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:04:57.411714 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-356bd6f8-fc41-463d-98e9-8e07b5a5ff4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:04:57.432388 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:04:57.519701 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356bd6f8-fc41-463d-98e9-8e07b5a5ff4c None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:04:57.592913 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b1e2b6-eb60-4313-913d-02e1028a5d68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:04:57.593514 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b1e2b6-eb60-4313-913d-02e1028a5d68 service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:04:57.593790 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b1e2b6-eb60-4313-913d-02e1028a5d68 service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:04:57.594058 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b1e2b6-eb60-4313-913d-02e1028a5d68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:04:57.596003 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-91b1e2b6-eb60-4313-913d-02e1028a5d68 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:04:57.670194 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:04:57.710548 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b1e2b6-eb60-4313-913d-02e1028a5d68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:04:57.712079 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b1e2b6-eb60-4313-913d-02e1028a5d68 service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:04:57.730582 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 656/1323] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:04:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:04:59.971377 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a49245-62a3-4182-838c-2dbc0cf458f3 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:04:59.971903 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a49245-62a3-4182-838c-2dbc0cf458f3 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:04:59.971903 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a49245-62a3-4182-838c-2dbc0cf458f3 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:04:59.973220 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 668/1324] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:04:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:04:59.978649 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5097328-0b1c-4dc1-9d62-3065c5e31a4d None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:04:59.979113 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5097328-0b1c-4dc1-9d62-3065c5e31a4d None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:04:59.979419 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5097328-0b1c-4dc1-9d62-3065c5e31a4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:05:00.046349 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-a5097328-0b1c-4dc1-9d62-3065c5e31a4d None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:05:00.055252 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-a5097328-0b1c-4dc1-9d62-3065c5e31a4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:05:00.153281 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 657/1325] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:04:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:05:00.434834 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a49245-62a3-4182-838c-2dbc0cf458f3 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:05:00.436488 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-59a49245-62a3-4182-838c-2dbc0cf458f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:05:00.541011 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a49245-62a3-4182-838c-2dbc0cf458f3 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:05:00.615706 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d122ec-b100-4e04-a12b-88eebb15ff11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:05:00.616231 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d122ec-b100-4e04-a12b-88eebb15ff11 service neutron] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:05:00.616511 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d122ec-b100-4e04-a12b-88eebb15ff11 service neutron] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:05:00.616775 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d122ec-b100-4e04-a12b-88eebb15ff11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:05:00.618361 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-90d122ec-b100-4e04-a12b-88eebb15ff11 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:05:00.636204 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:05:00.736100 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d122ec-b100-4e04-a12b-88eebb15ff11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:05:00.737521 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d122ec-b100-4e04-a12b-88eebb15ff11 service neutron] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:05:00.755655 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 669/1326] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:05:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:05:02.997068 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5142073f-8536-4665-91df-24352a21d25c None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:05:02.997559 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5142073f-8536-4665-91df-24352a21d25c None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:05:02.997559 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5142073f-8536-4665-91df-24352a21d25c None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:05:02.998883 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 658/1327] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:05:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:05:03.004966 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acca79cc-b8b2-4499-9f4e-13ec294b35dc None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:05:03.005382 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acca79cc-b8b2-4499-9f4e-13ec294b35dc None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:05:03.005517 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acca79cc-b8b2-4499-9f4e-13ec294b35dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:05:03.074157 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-acca79cc-b8b2-4499-9f4e-13ec294b35dc None None] MFA Rules not processed for user `1ccfba61dc7d4ad98da496e5800a8589`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:05:03.075481 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-acca79cc-b8b2-4499-9f4e-13ec294b35dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:05:03.140402 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:05:03.173104 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 670/1328] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:05:03 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:05:03.461721 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5142073f-8536-4665-91df-24352a21d25c None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:05:03.463317 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-5142073f-8536-4665-91df-24352a21d25c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:05:03.496308 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:05:03.572814 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5142073f-8536-4665-91df-24352a21d25c None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:05:03.651109 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3984a5d9-9d4d-4505-bf73-f64ad238e0b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:05:03.651719 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3984a5d9-9d4d-4505-bf73-f64ad238e0b9 service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:05:03.651990 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3984a5d9-9d4d-4505-bf73-f64ad238e0b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:05:03.652254 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3984a5d9-9d4d-4505-bf73-f64ad238e0b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:05:03.653833 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-3984a5d9-9d4d-4505-bf73-f64ad238e0b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:05:03.731538 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:05:03.763142 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3984a5d9-9d4d-4505-bf73-f64ad238e0b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:05:03.764515 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3984a5d9-9d4d-4505-bf73-f64ad238e0b9 service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:05:03.783464 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 659/1329] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:05:03 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:05:06.195843 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59098fdc-ca5b-4d31-869d-c4fbe514315b None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:05:06.196413 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59098fdc-ca5b-4d31-869d-c4fbe514315b None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:05:06.196413 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59098fdc-ca5b-4d31-869d-c4fbe514315b None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:05:06.197712 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 671/1330] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:05:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:05:06.203012 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b741d9b-6c68-444f-a4ad-9f9a8e1188bc None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:05:06.204056 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b741d9b-6c68-444f-a4ad-9f9a8e1188bc None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:05:06.204305 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b741d9b-6c68-444f-a4ad-9f9a8e1188bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:05:06.273424 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-0b741d9b-6c68-444f-a4ad-9f9a8e1188bc None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:05:06.274641 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-0b741d9b-6c68-444f-a4ad-9f9a8e1188bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:05:06.370872 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 660/1331] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:05:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:05:06.663881 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59098fdc-ca5b-4d31-869d-c4fbe514315b None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:05:06.665405 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-59098fdc-ca5b-4d31-869d-c4fbe514315b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:05:06.767244 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59098fdc-ca5b-4d31-869d-c4fbe514315b None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:05:06.840928 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c2511e-5f19-4895-8c5b-3a59c94c9c0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:05:06.841526 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c2511e-5f19-4895-8c5b-3a59c94c9c0c service neutron] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:05:06.841805 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c2511e-5f19-4895-8c5b-3a59c94c9c0c service neutron] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:05:06.842074 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c2511e-5f19-4895-8c5b-3a59c94c9c0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:05:06.844206 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-b5c2511e-5f19-4895-8c5b-3a59c94c9c0c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:05:06.871186 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:05:06.951872 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c2511e-5f19-4895-8c5b-3a59c94c9c0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:05:06.953392 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c2511e-5f19-4895-8c5b-3a59c94c9c0c service neutron] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:05:06.972146 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 672/1332] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:05:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:05:09.183547 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1231e89-5259-40e2-8fdc-38e43ee0f032 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:05:09.184084 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1231e89-5259-40e2-8fdc-38e43ee0f032 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:05:09.184084 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1231e89-5259-40e2-8fdc-38e43ee0f032 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:05:09.185432 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 661/1333] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:05:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:05:09.191299 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb34948-26f1-4879-abe0-d69bc655c6fe None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:05:09.191706 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb34948-26f1-4879-abe0-d69bc655c6fe None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:05:09.191840 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb34948-26f1-4879-abe0-d69bc655c6fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:05:09.256337 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-eeb34948-26f1-4879-abe0-d69bc655c6fe None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:05:09.257528 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-eeb34948-26f1-4879-abe0-d69bc655c6fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:05:09.330012 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:05:09.351820 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 673/1334] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:05:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:05:09.635252 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1231e89-5259-40e2-8fdc-38e43ee0f032 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:05:09.636788 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-c1231e89-5259-40e2-8fdc-38e43ee0f032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:05:09.677314 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:05:09.740706 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1231e89-5259-40e2-8fdc-38e43ee0f032 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:05:09.816991 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031536e7-d7db-492e-841c-c9ed29dab3c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:05:09.817574 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031536e7-d7db-492e-841c-c9ed29dab3c3 service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:05:09.817912 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031536e7-d7db-492e-841c-c9ed29dab3c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:05:09.818131 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031536e7-d7db-492e-841c-c9ed29dab3c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:05:09.819786 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-031536e7-d7db-492e-841c-c9ed29dab3c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:05:09.901119 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:05:09.927210 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031536e7-d7db-492e-841c-c9ed29dab3c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:05:09.928645 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031536e7-d7db-492e-841c-c9ed29dab3c3 service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:05:09.946919 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 662/1335] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:05:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:05:12.432572 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03119d0e-955a-4914-832b-9c45a1911fbc None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:05:12.433101 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03119d0e-955a-4914-832b-9c45a1911fbc None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:05:12.433406 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03119d0e-955a-4914-832b-9c45a1911fbc None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:05:12.434760 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 674/1336] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:05:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:05:12.440387 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cce2832-3809-4c87-a17f-c45b6b53bba9 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:05:12.440824 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cce2832-3809-4c87-a17f-c45b6b53bba9 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:05:12.440921 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cce2832-3809-4c87-a17f-c45b6b53bba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:05:12.507087 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-0cce2832-3809-4c87-a17f-c45b6b53bba9 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:05:12.508297 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-0cce2832-3809-4c87-a17f-c45b6b53bba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:05:12.604038 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 663/1337] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:05:12 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:05:12.888835 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03119d0e-955a-4914-832b-9c45a1911fbc None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:05:12.890511 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-03119d0e-955a-4914-832b-9c45a1911fbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:05:12.992365 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03119d0e-955a-4914-832b-9c45a1911fbc None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:05:13.065753 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58675c4c-3d30-4509-a8ae-5ada4b46b784 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:05:13.066352 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58675c4c-3d30-4509-a8ae-5ada4b46b784 service neutron] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:05:13.066631 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58675c4c-3d30-4509-a8ae-5ada4b46b784 service neutron] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:05:13.066927 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58675c4c-3d30-4509-a8ae-5ada4b46b784 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:05:13.068520 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-58675c4c-3d30-4509-a8ae-5ada4b46b784 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:05:13.103899 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:05:13.177103 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58675c4c-3d30-4509-a8ae-5ada4b46b784 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:05:13.178463 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58675c4c-3d30-4509-a8ae-5ada4b46b784 service neutron] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:05:13.197175 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 675/1338] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:05:12 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:05:17.293043 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14985768-203e-4e52-b147-61c8e8f5e36c None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:05:17.293627 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14985768-203e-4e52-b147-61c8e8f5e36c None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:05:17.293627 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14985768-203e-4e52-b147-61c8e8f5e36c None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:05:17.295008 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 664/1339] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:05:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:05:17.300462 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a313b5-6a9a-4664-9fa4-a01c61845abc None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:05:17.300894 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a313b5-6a9a-4664-9fa4-a01c61845abc None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:05:17.301083 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a313b5-6a9a-4664-9fa4-a01c61845abc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:05:17.367765 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-33a313b5-6a9a-4664-9fa4-a01c61845abc None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:05:17.368890 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-33a313b5-6a9a-4664-9fa4-a01c61845abc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:05:17.454339 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:05:17.466051 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 676/1340] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:05:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:05:17.768240 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3f25a7-792a-4c45-b64c-4c5ff1c177d2 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:05:17.768642 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3f25a7-792a-4c45-b64c-4c5ff1c177d2 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:05:17.768983 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3f25a7-792a-4c45-b64c-4c5ff1c177d2 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:05:17.770463 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 665/1341] 10.222.0.49 () {54 vars in 965 bytes} [Fri Sep 9 12:05:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:05:17.776409 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a34d50-503f-4f8e-9d1f-fe612d90e2a6 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:05:17.776696 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a34d50-503f-4f8e-9d1f-fe612d90e2a6 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:05:17.776968 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a34d50-503f-4f8e-9d1f-fe612d90e2a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:05:17.853363 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-69a34d50-503f-4f8e-9d1f-fe612d90e2a6 None None] MFA Rules not processed for user `992de6f86b42495dafe73cb2c393fb4e`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:05:17.854812 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-69a34d50-503f-4f8e-9d1f-fe612d90e2a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:05:17.931506 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 677/1342] 10.222.0.49 () {58 vars in 1062 bytes} [Fri Sep 9 12:05:17 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:05:17.941484 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3f25a7-792a-4c45-b64c-4c5ff1c177d2 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:05:17.943102 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-6d3f25a7-792a-4c45-b64c-4c5ff1c177d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:05:17.991602 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:05:18.047408 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3f25a7-792a-4c45-b64c-4c5ff1c177d2 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:05:18.123973 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5934de7-9522-4b07-829f-8c186466432d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:05:18.124738 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5934de7-9522-4b07-829f-8c186466432d service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:05:18.125039 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5934de7-9522-4b07-829f-8c186466432d service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:05:18.125311 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5934de7-9522-4b07-829f-8c186466432d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:05:18.126931 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-b5934de7-9522-4b07-829f-8c186466432d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:05:18.228875 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:05:18.246469 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5934de7-9522-4b07-829f-8c186466432d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:05:18.247965 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5934de7-9522-4b07-829f-8c186466432d service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:05:18.269057 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 666/1343] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:05:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:05:21.783196 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7239214-2fbc-473f-a5fa-a6a70a647c77 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:05:21.783698 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7239214-2fbc-473f-a5fa-a6a70a647c77 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:05:21.783790 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7239214-2fbc-473f-a5fa-a6a70a647c77 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:05:21.785091 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 678/1344] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:05:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:05:21.790237 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decbb4ba-c8b8-4d2c-9824-47f39a585c90 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:05:21.790640 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decbb4ba-c8b8-4d2c-9824-47f39a585c90 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:05:21.790841 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decbb4ba-c8b8-4d2c-9824-47f39a585c90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:05:21.857470 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-decbb4ba-c8b8-4d2c-9824-47f39a585c90 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:05:21.859011 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-decbb4ba-c8b8-4d2c-9824-47f39a585c90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:05:21.977227 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 667/1345] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:05:21 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:05:22.273871 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7239214-2fbc-473f-a5fa-a6a70a647c77 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:05:22.275448 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-d7239214-2fbc-473f-a5fa-a6a70a647c77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:05:22.364585 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:05:22.383787 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7239214-2fbc-473f-a5fa-a6a70a647c77 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:05:22.461514 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbe7d57-9b2d-4f51-b073-274dc02d3a46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:05:22.462129 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbe7d57-9b2d-4f51-b073-274dc02d3a46 service neutron] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:05:22.462405 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbe7d57-9b2d-4f51-b073-274dc02d3a46 service neutron] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:05:22.462670 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbe7d57-9b2d-4f51-b073-274dc02d3a46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:05:22.464333 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-6dbe7d57-9b2d-4f51-b073-274dc02d3a46 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:05:22.579065 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbe7d57-9b2d-4f51-b073-274dc02d3a46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:05:22.580565 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbe7d57-9b2d-4f51-b073-274dc02d3a46 service neutron] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:05:22.600317 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 679/1346] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:05:22 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:05:26.942510 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c23956-44b1-4960-a6db-d56b9390f2f9 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:05:26.943431 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c23956-44b1-4960-a6db-d56b9390f2f9 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:05:26.943726 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c23956-44b1-4960-a6db-d56b9390f2f9 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:05:26.945128 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 668/1347] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:05:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 12:05:26.950653 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9522b1f6-3ade-47cd-918a-3f005de15b5e None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:05:26.951131 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9522b1f6-3ade-47cd-918a-3f005de15b5e None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:05:26.951270 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9522b1f6-3ade-47cd-918a-3f005de15b5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:05:26.969035 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:05:27.022500 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-9522b1f6-3ade-47cd-918a-3f005de15b5e None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:05:27.023753 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-9522b1f6-3ade-47cd-918a-3f005de15b5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:05:27.124761 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 680/1348] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:05:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:05:27.408858 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c23956-44b1-4960-a6db-d56b9390f2f9 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:05:27.410408 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-d2c23956-44b1-4960-a6db-d56b9390f2f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:05:27.467714 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:05:27.512728 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c23956-44b1-4960-a6db-d56b9390f2f9 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:05:27.584828 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41901e01-1e03-4ff4-ad23-0e5cdcb2a0b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:05:27.585421 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41901e01-1e03-4ff4-ad23-0e5cdcb2a0b9 service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:05:27.585693 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41901e01-1e03-4ff4-ad23-0e5cdcb2a0b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:05:27.585960 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41901e01-1e03-4ff4-ad23-0e5cdcb2a0b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:05:27.587559 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-41901e01-1e03-4ff4-ad23-0e5cdcb2a0b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:05:27.693917 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:05:27.698103 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41901e01-1e03-4ff4-ad23-0e5cdcb2a0b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:05:27.699578 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41901e01-1e03-4ff4-ad23-0e5cdcb2a0b9 service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:05:27.717173 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 669/1349] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:05:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:05:32.690943 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641c3ed9-3f80-4afe-a99f-06182122e174 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:05:32.691372 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641c3ed9-3f80-4afe-a99f-06182122e174 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:05:32.691555 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641c3ed9-3f80-4afe-a99f-06182122e174 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:05:32.692841 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 681/1350] 10.222.0.49 () {54 vars in 921 bytes} [Fri Sep 9 12:05:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:05:32.698075 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee53ba8-0c29-4b63-8bd3-7a9042da7e76 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:05:32.698520 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee53ba8-0c29-4b63-8bd3-7a9042da7e76 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:05:32.698604 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee53ba8-0c29-4b63-8bd3-7a9042da7e76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:05:32.791683 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-fee53ba8-0c29-4b63-8bd3-7a9042da7e76 None None] MFA Rules not processed for user `d3adb2f0583c4ff782049dacdb3ee556`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:05:32.793283 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-fee53ba8-0c29-4b63-8bd3-7a9042da7e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:05:32.950655 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 670/1351] 10.222.0.49 () {58 vars in 1018 bytes} [Fri Sep 9 12:05:32 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:05:37.017093 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef17aea5-0c7f-4ffa-9ba7-fcdf0c950492 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:05:37.017607 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef17aea5-0c7f-4ffa-9ba7-fcdf0c950492 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:05:37.017607 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef17aea5-0c7f-4ffa-9ba7-fcdf0c950492 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:05:37.019114 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 682/1352] 10.222.0.49 () {54 vars in 921 bytes} [Fri Sep 9 12:05:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:05:37.024016 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b886b9-2ae1-46f6-92e4-4c71213fba75 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:05:37.024461 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b886b9-2ae1-46f6-92e4-4c71213fba75 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:05:37.024545 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b886b9-2ae1-46f6-92e4-4c71213fba75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:05:37.081457 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:05:37.108806 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-73b886b9-2ae1-46f6-92e4-4c71213fba75 None None] MFA Rules not processed for user `d3adb2f0583c4ff782049dacdb3ee556`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:05:37.110025 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-73b886b9-2ae1-46f6-92e4-4c71213fba75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:05:37.123732 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=59 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:05:37.224181 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 671/1353] 10.222.0.49 () {58 vars in 1018 bytes} [Fri Sep 9 12:05:37 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:05:38.724499 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d769f8-976c-43b3-a4f2-2497bce4f730 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:05:38.725067 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d769f8-976c-43b3-a4f2-2497bce4f730 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:05:38.725067 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d769f8-976c-43b3-a4f2-2497bce4f730 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:05:38.726374 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 683/1354] 10.222.0.49 () {54 vars in 921 bytes} [Fri Sep 9 12:05:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:05:38.731651 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46769702-88f0-47fe-b2b5-a70a0a4506f5 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:05:38.732562 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46769702-88f0-47fe-b2b5-a70a0a4506f5 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:05:38.732964 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46769702-88f0-47fe-b2b5-a70a0a4506f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:05:38.812485 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-46769702-88f0-47fe-b2b5-a70a0a4506f5 None None] MFA Rules not processed for user `d3adb2f0583c4ff782049dacdb3ee556`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:05:38.813675 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-46769702-88f0-47fe-b2b5-a70a0a4506f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:05:38.855202 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:05:38.926068 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 672/1355] 10.222.0.49 () {58 vars in 1018 bytes} [Fri Sep 9 12:05:38 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:05:42.467201 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baf9470-a8b7-4b30-aaff-e047f7f6b037 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:05:42.467698 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baf9470-a8b7-4b30-aaff-e047f7f6b037 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:05:42.467877 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baf9470-a8b7-4b30-aaff-e047f7f6b037 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:05:42.469327 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 684/1356] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:05:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:05:42.474883 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e581f849-144e-47e9-a499-5e518ca0f81c None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:05:42.475376 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e581f849-144e-47e9-a499-5e518ca0f81c None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:05:42.475670 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e581f849-144e-47e9-a499-5e518ca0f81c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:05:42.547856 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-e581f849-144e-47e9-a499-5e518ca0f81c None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:05:42.549111 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-e581f849-144e-47e9-a499-5e518ca0f81c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:05:42.656923 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 673/1357] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:05:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:05:42.966529 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384bcba9-399b-4674-afe0-336886f478d3 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:05:42.966934 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384bcba9-399b-4674-afe0-336886f478d3 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:05:42.967316 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384bcba9-399b-4674-afe0-336886f478d3 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:05:42.968754 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 685/1358] 10.222.0.49 () {54 vars in 962 bytes} [Fri Sep 9 12:05:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:05:42.975882 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc775443-63bc-417e-8152-00530aa8d1b7 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:05:42.976167 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc775443-63bc-417e-8152-00530aa8d1b7 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:05:42.976439 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc775443-63bc-417e-8152-00530aa8d1b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:05:43.002400 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:05:43.067694 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-dc775443-63bc-417e-8152-00530aa8d1b7 None None] MFA Rules not processed for user `70bac169d5cd43e1926f310344ac11fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:05:43.068893 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-dc775443-63bc-417e-8152-00530aa8d1b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:05:43.205396 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 674/1359] 10.222.0.49 () {58 vars in 1059 bytes} [Fri Sep 9 12:05:42 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:05:43.212779 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384bcba9-399b-4674-afe0-336886f478d3 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:05:43.214285 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-384bcba9-399b-4674-afe0-336886f478d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:05:43.375917 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384bcba9-399b-4674-afe0-336886f478d3 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:05:43.452578 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:05:43.493970 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2afbdfb-5d86-42e3-a221-00c2e1617008 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70bac169d5cd43e1926f310344ac11fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:05:43.494607 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2afbdfb-5d86-42e3-a221-00c2e1617008 service nova] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:05:43.494941 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2afbdfb-5d86-42e3-a221-00c2e1617008 service nova] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:05:43.495276 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2afbdfb-5d86-42e3-a221-00c2e1617008 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:05:43.496941 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-c2afbdfb-5d86-42e3-a221-00c2e1617008 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:05:43.598673 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2afbdfb-5d86-42e3-a221-00c2e1617008 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:05:43.600438 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2afbdfb-5d86-42e3-a221-00c2e1617008 service nova] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:05:43.620058 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 686/1360] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:05:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:05:46.064403 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667e44a5-594a-4b37-ab84-6d70d5d42fc1 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:05:46.065069 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667e44a5-594a-4b37-ab84-6d70d5d42fc1 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:05:46.065069 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667e44a5-594a-4b37-ab84-6d70d5d42fc1 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:05:46.075246 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 675/1361] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:05:46 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:05:46.075446 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b7e49d-a4af-4da2-95c0-8768409f90cb None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:05:46.075446 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b7e49d-a4af-4da2-95c0-8768409f90cb None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:05:46.075446 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b7e49d-a4af-4da2-95c0-8768409f90cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:05:46.167773 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-26b7e49d-a4af-4da2-95c0-8768409f90cb None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:05:46.169828 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-26b7e49d-a4af-4da2-95c0-8768409f90cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:05:46.210263 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:05:46.270727 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 687/1362] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:05:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:05:46.567398 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667e44a5-594a-4b37-ab84-6d70d5d42fc1 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:05:46.568903 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-667e44a5-594a-4b37-ab84-6d70d5d42fc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:05:46.654529 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:05:46.767547 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667e44a5-594a-4b37-ab84-6d70d5d42fc1 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:05:46.902939 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63993043-4509-4380-95e5-d550421e7f8c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70bac169d5cd43e1926f310344ac11fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:05:46.904104 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63993043-4509-4380-95e5-d550421e7f8c service nova] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:05:46.904386 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63993043-4509-4380-95e5-d550421e7f8c service nova] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:05:46.904656 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63993043-4509-4380-95e5-d550421e7f8c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:05:46.906284 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-63993043-4509-4380-95e5-d550421e7f8c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:05:46.930871 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:05:47.002184 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63993043-4509-4380-95e5-d550421e7f8c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:05:47.003819 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63993043-4509-4380-95e5-d550421e7f8c service nova] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:05:47.022638 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 676/1363] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:05:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:05:49.554434 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eca1698-b342-419f-bf6c-47f6f4084682 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:05:49.555035 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eca1698-b342-419f-bf6c-47f6f4084682 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:05:49.555035 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eca1698-b342-419f-bf6c-47f6f4084682 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:05:49.556524 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 688/1364] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:05:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:05:49.562287 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af994896-511a-434d-b77c-202bb58e280a None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:05:49.562708 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af994896-511a-434d-b77c-202bb58e280a None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:05:49.562882 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af994896-511a-434d-b77c-202bb58e280a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:05:49.630612 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-af994896-511a-434d-b77c-202bb58e280a None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:05:49.632010 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-af994896-511a-434d-b77c-202bb58e280a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:05:49.737148 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:05:49.750686 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 677/1365] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:05:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:05:50.029066 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebaa100-c6fc-436c-8df9-df619d8fe375 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:05:50.029355 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebaa100-c6fc-436c-8df9-df619d8fe375 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:05:50.029676 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebaa100-c6fc-436c-8df9-df619d8fe375 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:05:50.031205 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 689/1366] 10.222.0.49 () {54 vars in 962 bytes} [Fri Sep 9 12:05:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:05:50.036483 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9024469e-5d68-4f1c-bc65-2b032676f872 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:05:50.036837 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9024469e-5d68-4f1c-bc65-2b032676f872 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:05:50.037184 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9024469e-5d68-4f1c-bc65-2b032676f872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:05:50.119310 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-9024469e-5d68-4f1c-bc65-2b032676f872 None None] MFA Rules not processed for user `70bac169d5cd43e1926f310344ac11fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:05:50.120518 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-9024469e-5d68-4f1c-bc65-2b032676f872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:05:50.243427 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 678/1367] 10.222.0.49 () {58 vars in 1059 bytes} [Fri Sep 9 12:05:50 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:05:50.251965 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebaa100-c6fc-436c-8df9-df619d8fe375 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:05:50.253458 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-eebaa100-c6fc-436c-8df9-df619d8fe375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:05:50.409460 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebaa100-c6fc-436c-8df9-df619d8fe375 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:05:50.431037 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:05:50.533905 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc56194-8e8c-420c-8ecf-073d3884f162 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70bac169d5cd43e1926f310344ac11fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:05:50.534522 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc56194-8e8c-420c-8ecf-073d3884f162 service nova] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:05:50.534796 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc56194-8e8c-420c-8ecf-073d3884f162 service nova] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:05:50.535090 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc56194-8e8c-420c-8ecf-073d3884f162 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:05:50.536684 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-0fc56194-8e8c-420c-8ecf-073d3884f162 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:05:50.631600 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc56194-8e8c-420c-8ecf-073d3884f162 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:05:50.633139 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc56194-8e8c-420c-8ecf-073d3884f162 service nova] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:05:50.660677 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 690/1368] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:05:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:05:50.766564 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb78ca7-a5d8-4f5d-b9d2-52e1d002a4a2 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:05:50.767161 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb78ca7-a5d8-4f5d-b9d2-52e1d002a4a2 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:05:50.767323 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb78ca7-a5d8-4f5d-b9d2-52e1d002a4a2 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:05:50.768890 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 679/1369] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:05:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:05:50.774423 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17841f1-4987-426a-83e6-82beb3e64cc3 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:05:50.774788 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17841f1-4987-426a-83e6-82beb3e64cc3 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:05:50.775185 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17841f1-4987-426a-83e6-82beb3e64cc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:05:50.815893 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:05:50.862246 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-a17841f1-4987-426a-83e6-82beb3e64cc3 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:05:50.863875 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-a17841f1-4987-426a-83e6-82beb3e64cc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:05:50.985302 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 691/1370] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:05:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:05:51.322422 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8ed89b-b216-460c-81d8-6a826b3be5a7 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:05:51.322422 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8ed89b-b216-460c-81d8-6a826b3be5a7 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:05:51.322821 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8ed89b-b216-460c-81d8-6a826b3be5a7 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:05:51.323974 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 680/1371] 10.222.0.49 () {54 vars in 964 bytes} [Fri Sep 9 12:05:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:05:51.329815 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3e684c-e5af-416d-a44b-6a3f34c7b5ca None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:05:51.330068 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3e684c-e5af-416d-a44b-6a3f34c7b5ca None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:05:51.330337 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3e684c-e5af-416d-a44b-6a3f34c7b5ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:05:51.416191 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-dc3e684c-e5af-416d-a44b-6a3f34c7b5ca None None] MFA Rules not processed for user `535c258100df40b58a5ee44b2c21d621`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:05:51.417530 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-dc3e684c-e5af-416d-a44b-6a3f34c7b5ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:05:51.468919 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:05:51.504360 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 692/1372] 10.222.0.49 () {58 vars in 1061 bytes} [Fri Sep 9 12:05:51 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:05:51.511905 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8ed89b-b216-460c-81d8-6a826b3be5a7 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:05:51.514444 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-aa8ed89b-b216-460c-81d8-6a826b3be5a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:05:51.568245 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:05:51.627204 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8ed89b-b216-460c-81d8-6a826b3be5a7 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:05:51.735180 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c12b0e-566f-40f1-8c60-805925441f61 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535c258100df40b58a5ee44b2c21d621', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:05:51.736069 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c12b0e-566f-40f1-8c60-805925441f61 service cinder] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:05:51.736526 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c12b0e-566f-40f1-8c60-805925441f61 service cinder] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:05:51.737015 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c12b0e-566f-40f1-8c60-805925441f61 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:05:51.739127 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-11c12b0e-566f-40f1-8c60-805925441f61 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:05:51.882186 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:05:51.906714 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c12b0e-566f-40f1-8c60-805925441f61 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:05:51.908917 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c12b0e-566f-40f1-8c60-805925441f61 service cinder] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:05:51.937925 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 681/1373] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:05:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:05:52.974485 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e70d81e-9cab-42bf-80fd-7bda96827b92 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:05:52.975094 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e70d81e-9cab-42bf-80fd-7bda96827b92 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:05:52.975094 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e70d81e-9cab-42bf-80fd-7bda96827b92 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:05:52.976291 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 693/1374] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:05:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:05:52.982259 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73439ab0-ce05-4baf-b2fa-6ceee81e5c24 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:05:52.982684 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73439ab0-ce05-4baf-b2fa-6ceee81e5c24 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:05:52.982816 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73439ab0-ce05-4baf-b2fa-6ceee81e5c24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:05:53.048088 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-73439ab0-ce05-4baf-b2fa-6ceee81e5c24 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:05:53.049297 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-73439ab0-ce05-4baf-b2fa-6ceee81e5c24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:05:53.146882 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 682/1375] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:05:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:05:53.444020 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e70d81e-9cab-42bf-80fd-7bda96827b92 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:05:53.445711 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-0e70d81e-9cab-42bf-80fd-7bda96827b92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:05:53.611822 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e70d81e-9cab-42bf-80fd-7bda96827b92 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:05:53.669330 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:05:53.736666 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3e128b-24f0-4c2c-96aa-e93be0ae2340 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70bac169d5cd43e1926f310344ac11fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:05:53.737271 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3e128b-24f0-4c2c-96aa-e93be0ae2340 service nova] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:05:53.737548 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3e128b-24f0-4c2c-96aa-e93be0ae2340 service nova] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:05:53.737819 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3e128b-24f0-4c2c-96aa-e93be0ae2340 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:05:53.739462 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-ad3e128b-24f0-4c2c-96aa-e93be0ae2340 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:05:53.853350 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3e128b-24f0-4c2c-96aa-e93be0ae2340 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:05:53.854997 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3e128b-24f0-4c2c-96aa-e93be0ae2340 service nova] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:05:53.873965 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 694/1376] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:05:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:05:56.083255 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c4f0ef-8ad4-4bc8-a411-213eb4a08776 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:05:56.083799 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c4f0ef-8ad4-4bc8-a411-213eb4a08776 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:05:56.083799 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c4f0ef-8ad4-4bc8-a411-213eb4a08776 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:05:56.085283 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 683/1377] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:05:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:05:56.090993 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cbd416-9790-46ef-bdf9-a2f5841d2b2d None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:05:56.091393 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cbd416-9790-46ef-bdf9-a2f5841d2b2d None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:05:56.091611 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cbd416-9790-46ef-bdf9-a2f5841d2b2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:05:56.161916 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-85cbd416-9790-46ef-bdf9-a2f5841d2b2d None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:05:56.163138 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-85cbd416-9790-46ef-bdf9-a2f5841d2b2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:05:56.170482 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:05:56.269267 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 695/1378] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:05:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:05:56.559748 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c4f0ef-8ad4-4bc8-a411-213eb4a08776 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:05:56.561493 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-55c4f0ef-8ad4-4bc8-a411-213eb4a08776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:05:56.633248 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:05:56.731172 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c4f0ef-8ad4-4bc8-a411-213eb4a08776 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:05:56.861813 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13eb8bd-58c0-49f1-b454-c021af425b92 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70bac169d5cd43e1926f310344ac11fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:05:56.862429 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13eb8bd-58c0-49f1-b454-c021af425b92 service nova] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:05:56.862707 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13eb8bd-58c0-49f1-b454-c021af425b92 service nova] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:05:56.863133 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13eb8bd-58c0-49f1-b454-c021af425b92 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:05:56.864718 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-a13eb8bd-58c0-49f1-b454-c021af425b92 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:05:56.885888 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:05:56.964343 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13eb8bd-58c0-49f1-b454-c021af425b92 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:05:56.966033 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13eb8bd-58c0-49f1-b454-c021af425b92 service nova] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:05:56.985993 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 684/1379] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:05:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:05:59.222222 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ef57c8-5646-456e-94ed-20903d44110e None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:05:59.222781 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ef57c8-5646-456e-94ed-20903d44110e None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:05:59.223102 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ef57c8-5646-456e-94ed-20903d44110e None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:05:59.224736 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 696/1380] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:05:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:05:59.229833 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c09a21-95b9-4fee-820a-08902af96ac4 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:05:59.230873 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c09a21-95b9-4fee-820a-08902af96ac4 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:05:59.230994 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c09a21-95b9-4fee-820a-08902af96ac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:05:59.301083 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-64c09a21-95b9-4fee-820a-08902af96ac4 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:05:59.302364 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-64c09a21-95b9-4fee-820a-08902af96ac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:05:59.423930 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:05:59.431140 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 685/1381] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:05:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:05:59.726532 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ef57c8-5646-456e-94ed-20903d44110e None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:05:59.728029 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-f2ef57c8-5646-456e-94ed-20903d44110e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:05:59.890309 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:05:59.901934 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ef57c8-5646-456e-94ed-20903d44110e None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:00.053246 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cd9485-55c2-4172-9d84-848f54cca4c1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70bac169d5cd43e1926f310344ac11fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:00.054365 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cd9485-55c2-4172-9d84-848f54cca4c1 service nova] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:00.054817 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cd9485-55c2-4172-9d84-848f54cca4c1 service nova] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:00.055226 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cd9485-55c2-4172-9d84-848f54cca4c1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:00.057525 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-71cd9485-55c2-4172-9d84-848f54cca4c1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:00.174531 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cd9485-55c2-4172-9d84-848f54cca4c1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:00.176870 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cd9485-55c2-4172-9d84-848f54cca4c1 service nova] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:00.191504 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:00.201787 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 697/1382] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:05:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:06:02.416096 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd3bb54-7ee4-4ae4-83e2-eaeceb2c048e None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:02.416619 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd3bb54-7ee4-4ae4-83e2-eaeceb2c048e None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:02.416619 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd3bb54-7ee4-4ae4-83e2-eaeceb2c048e None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:02.418881 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 686/1383] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:06:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 12:06:02.424297 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760016d5-a83e-4962-b2a9-afead9cb9fa2 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:02.424748 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760016d5-a83e-4962-b2a9-afead9cb9fa2 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:02.424870 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760016d5-a83e-4962-b2a9-afead9cb9fa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:02.494789 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-760016d5-a83e-4962-b2a9-afead9cb9fa2 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:06:02.496565 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-760016d5-a83e-4962-b2a9-afead9cb9fa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:02.607856 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 698/1384] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:06:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:06:02.904212 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd3bb54-7ee4-4ae4-83e2-eaeceb2c048e None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:02.905734 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-6dd3bb54-7ee4-4ae4-83e2-eaeceb2c048e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:03.084298 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd3bb54-7ee4-4ae4-83e2-eaeceb2c048e None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:03.159975 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:03.208842 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd82c92-3d2f-40da-81f3-5f61f39b77a2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70bac169d5cd43e1926f310344ac11fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:03.209447 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd82c92-3d2f-40da-81f3-5f61f39b77a2 service nova] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:03.209723 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd82c92-3d2f-40da-81f3-5f61f39b77a2 service nova] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:03.210001 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd82c92-3d2f-40da-81f3-5f61f39b77a2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:03.211635 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-dfd82c92-3d2f-40da-81f3-5f61f39b77a2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:03.315810 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd82c92-3d2f-40da-81f3-5f61f39b77a2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:03.317498 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd82c92-3d2f-40da-81f3-5f61f39b77a2 service nova] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:03.337188 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 687/1385] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:06:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:06:05.535755 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a10110-61ea-43df-9abd-8e117da9c62c None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:05.536343 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a10110-61ea-43df-9abd-8e117da9c62c None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:05.536343 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a10110-61ea-43df-9abd-8e117da9c62c None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:05.537695 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 699/1386] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:06:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:06:05.544254 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9bce14-90f2-47b7-940c-96e67a99ea30 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:05.544690 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9bce14-90f2-47b7-940c-96e67a99ea30 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:05.544916 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9bce14-90f2-47b7-940c-96e67a99ea30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:05.611761 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-bd9bce14-90f2-47b7-940c-96e67a99ea30 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:06:05.612962 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-bd9bce14-90f2-47b7-940c-96e67a99ea30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:05.649759 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:05.716686 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 688/1387] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:06:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:06:06.006207 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a10110-61ea-43df-9abd-8e117da9c62c None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:06.007759 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-55a10110-61ea-43df-9abd-8e117da9c62c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:06.086753 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:06.157685 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a10110-61ea-43df-9abd-8e117da9c62c None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:06.267960 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953c32a2-4f5b-4f28-8865-a7971905c990 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70bac169d5cd43e1926f310344ac11fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:06.268565 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953c32a2-4f5b-4f28-8865-a7971905c990 service nova] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:06.268841 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953c32a2-4f5b-4f28-8865-a7971905c990 service nova] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:06.269107 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953c32a2-4f5b-4f28-8865-a7971905c990 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:06.270686 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-953c32a2-4f5b-4f28-8865-a7971905c990 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:06.310265 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:06.361331 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953c32a2-4f5b-4f28-8865-a7971905c990 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:06.362960 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953c32a2-4f5b-4f28-8865-a7971905c990 service nova] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:06.380935 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 700/1388] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:06:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:06:07.326232 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf542520-3a07-49e7-b12f-a46c92479640 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:07.326815 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf542520-3a07-49e7-b12f-a46c92479640 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:07.326815 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf542520-3a07-49e7-b12f-a46c92479640 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:07.328221 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 689/1389] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:06:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:06:07.333574 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6d9aa8-8c03-4e74-a139-82fec20f9e1a None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:07.333960 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6d9aa8-8c03-4e74-a139-82fec20f9e1a None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:07.334146 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6d9aa8-8c03-4e74-a139-82fec20f9e1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:07.401176 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-da6d9aa8-8c03-4e74-a139-82fec20f9e1a None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:06:07.402459 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-da6d9aa8-8c03-4e74-a139-82fec20f9e1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:07.507639 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 701/1390] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:06:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:06:07.815874 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5357bdde-c710-420b-882f-884a2ffa123d None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:07.816187 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5357bdde-c710-420b-882f-884a2ffa123d None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:07.816431 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5357bdde-c710-420b-882f-884a2ffa123d None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:07.817936 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 690/1391] 10.222.0.49 () {54 vars in 966 bytes} [Fri Sep 9 12:06:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:06:07.823434 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8ca01c-eee5-4585-a74d-66c066e16e36 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:07.823872 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8ca01c-eee5-4585-a74d-66c066e16e36 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:07.824409 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8ca01c-eee5-4585-a74d-66c066e16e36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:07.843184 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:07.910563 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-bb8ca01c-eee5-4585-a74d-66c066e16e36 None None] MFA Rules not processed for user `37c4e14aa8e04d3e85a406b9363bca2a`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:06:07.912156 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-bb8ca01c-eee5-4585-a74d-66c066e16e36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:07.998662 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 702/1392] 10.222.0.49 () {58 vars in 1063 bytes} [Fri Sep 9 12:06:07 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:06:08.007744 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5357bdde-c710-420b-882f-884a2ffa123d None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:08.009381 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-5357bdde-c710-420b-882f-884a2ffa123d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:08.119364 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5357bdde-c710-420b-882f-884a2ffa123d None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:08.185655 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:08.202079 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c844884-0193-4e2e-ae9e-965f437242a9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37c4e14aa8e04d3e85a406b9363bca2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:08.202722 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c844884-0193-4e2e-ae9e-965f437242a9 service glance] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:08.203033 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c844884-0193-4e2e-ae9e-965f437242a9 service glance] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:08.203307 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c844884-0193-4e2e-ae9e-965f437242a9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:08.205026 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-9c844884-0193-4e2e-ae9e-965f437242a9 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:08.320586 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c844884-0193-4e2e-ae9e-965f437242a9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:08.322033 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c844884-0193-4e2e-ae9e-965f437242a9 service glance] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:08.341150 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 691/1393] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:06:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:06:08.574596 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefabd5f-8013-431f-8c09-a4f81cde5ede None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:08.575191 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefabd5f-8013-431f-8c09-a4f81cde5ede None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:08.575191 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefabd5f-8013-431f-8c09-a4f81cde5ede None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:08.576649 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 703/1394] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:06:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:06:08.582719 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae369591-9ba4-406d-9db4-155207eb7d76 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:08.583033 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae369591-9ba4-406d-9db4-155207eb7d76 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:08.583315 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae369591-9ba4-406d-9db4-155207eb7d76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:08.654485 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-ae369591-9ba4-406d-9db4-155207eb7d76 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:06:08.655813 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-ae369591-9ba4-406d-9db4-155207eb7d76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:08.699747 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:08.765165 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 692/1395] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:06:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:06:09.052901 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefabd5f-8013-431f-8c09-a4f81cde5ede None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:09.054761 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-fefabd5f-8013-431f-8c09-a4f81cde5ede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:09.111896 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac51c14-f4c1-4a35-a5e3-dce242b31111 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:09.112181 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac51c14-f4c1-4a35-a5e3-dce242b31111 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:09.112454 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac51c14-f4c1-4a35-a5e3-dce242b31111 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:09.173354 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:09.180324 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-dac51c14-f4c1-4a35-a5e3-dce242b31111 None None] MFA Rules not processed for user `e368b4e668c54e25bff80a8e8a8c3588`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:06:09.181634 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-dac51c14-f4c1-4a35-a5e3-dce242b31111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:09.219736 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefabd5f-8013-431f-8c09-a4f81cde5ede None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:09.269280 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 693/1396] 10.222.0.49 () {58 vars in 1009 bytes} [Fri Sep 9 12:06:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:06:09.317342 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a47644-09dd-4d72-af75-1e1d443346e3 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:09.317711 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a47644-09dd-4d72-af75-1e1d443346e3 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:09.318061 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a47644-09dd-4d72-af75-1e1d443346e3 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:09.319750 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 694/1397] 10.222.0.49 () {54 vars in 943 bytes} [Fri Sep 9 12:06:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 12:06:09.327391 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf304be-dfe2-4f02-b047-56956a6782fd None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:09.327755 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf304be-dfe2-4f02-b047-56956a6782fd None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:09.328087 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf304be-dfe2-4f02-b047-56956a6782fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:09.339051 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6f5c70-f0d1-44d5-850d-c2d14878b0c3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70bac169d5cd43e1926f310344ac11fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:09.339660 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6f5c70-f0d1-44d5-850d-c2d14878b0c3 service nova] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:09.339928 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6f5c70-f0d1-44d5-850d-c2d14878b0c3 service nova] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:09.340196 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6f5c70-f0d1-44d5-850d-c2d14878b0c3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:09.341786 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-7c6f5c70-f0d1-44d5-850d-c2d14878b0c3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:09.373314 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:09.412050 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-faf304be-dfe2-4f02-b047-56956a6782fd None None] MFA Rules not processed for user `92e652ba827849b5a3fc1213d04a1b76`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:06:09.413280 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-faf304be-dfe2-4f02-b047-56956a6782fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:09.415321 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:09.436945 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6f5c70-f0d1-44d5-850d-c2d14878b0c3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:09.438554 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6f5c70-f0d1-44d5-850d-c2d14878b0c3 service nova] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:09.457590 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 704/1398] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:06:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:06:09.523663 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 695/1399] 10.222.0.49 () {58 vars in 1040 bytes} [Fri Sep 9 12:06:09 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:06:09.534212 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6f5c70-f0d1-44d5-850d-c2d14878b0c3 service nova] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:09.535573 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-7c6f5c70-f0d1-44d5-850d-c2d14878b0c3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:09.634410 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6f5c70-f0d1-44d5-850d-c2d14878b0c3 service nova] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:09.747033 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:09.748669 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc6ba29-4c80-4b91-a815-f371f63adc12 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e652ba827849b5a3fc1213d04a1b76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:09.749326 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc6ba29-4c80-4b91-a815-f371f63adc12 service swift] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:09.749692 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc6ba29-4c80-4b91-a815-f371f63adc12 service swift] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:09.750153 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc6ba29-4c80-4b91-a815-f371f63adc12 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:09.752229 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-6fc6ba29-4c80-4b91-a815-f371f63adc12 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:09.842343 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc6ba29-4c80-4b91-a815-f371f63adc12 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:09.843988 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc6ba29-4c80-4b91-a815-f371f63adc12 service swift] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:09.852216 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 705/1400] 10.222.0.49 () {60 vars in 1384 bytes} [Fri Sep 9 12:06:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 09 12:06:11.704166 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5c7e8e-ca12-4084-870a-9a42e105c717 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:11.704719 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5c7e8e-ca12-4084-870a-9a42e105c717 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:11.704719 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5c7e8e-ca12-4084-870a-9a42e105c717 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:11.706085 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 696/1401] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:06:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:06:11.711762 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182eae6e-3b84-4120-95d1-e48f053eae31 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:11.712188 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182eae6e-3b84-4120-95d1-e48f053eae31 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:11.712298 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182eae6e-3b84-4120-95d1-e48f053eae31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:11.792653 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-182eae6e-3b84-4120-95d1-e48f053eae31 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:06:11.793139 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-182eae6e-3b84-4120-95d1-e48f053eae31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:11.893035 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:11.905313 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 706/1402] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:06:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:06:12.208287 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5c7e8e-ca12-4084-870a-9a42e105c717 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:12.210241 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-9c5c7e8e-ca12-4084-870a-9a42e105c717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:12.320082 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:12.387857 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5c7e8e-ca12-4084-870a-9a42e105c717 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:12.515750 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2392f857-128f-434a-9467-b1bd0161d5e2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70bac169d5cd43e1926f310344ac11fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:12.516365 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2392f857-128f-434a-9467-b1bd0161d5e2 service nova] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:12.516644 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2392f857-128f-434a-9467-b1bd0161d5e2 service nova] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:12.516915 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2392f857-128f-434a-9467-b1bd0161d5e2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:12.518521 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-2392f857-128f-434a-9467-b1bd0161d5e2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:12.578808 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:12.613579 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2392f857-128f-434a-9467-b1bd0161d5e2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:12.615215 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2392f857-128f-434a-9467-b1bd0161d5e2 service nova] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:12.648101 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 697/1403] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:06:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:06:14.892607 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d592dd6d-f9bf-4059-a26a-82ff1818aeda None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:14.893202 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d592dd6d-f9bf-4059-a26a-82ff1818aeda None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:14.893202 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d592dd6d-f9bf-4059-a26a-82ff1818aeda None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:14.894499 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 707/1404] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:06:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:06:14.900268 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24854553-0420-4d00-a246-06a56aaa5d5a None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:14.900793 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24854553-0420-4d00-a246-06a56aaa5d5a None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:14.900993 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24854553-0420-4d00-a246-06a56aaa5d5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:14.978001 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-24854553-0420-4d00-a246-06a56aaa5d5a None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:06:14.979703 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-24854553-0420-4d00-a246-06a56aaa5d5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:15.091012 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 698/1405] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:06:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:06:15.383475 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d592dd6d-f9bf-4059-a26a-82ff1818aeda None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:15.385002 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-d592dd6d-f9bf-4059-a26a-82ff1818aeda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:15.540530 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d592dd6d-f9bf-4059-a26a-82ff1818aeda None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:15.628521 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:15.693912 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52decbbc-5a54-4292-b256-6561e66a000d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70bac169d5cd43e1926f310344ac11fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:15.694396 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52decbbc-5a54-4292-b256-6561e66a000d service nova] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:15.695071 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52decbbc-5a54-4292-b256-6561e66a000d service nova] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:15.695343 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52decbbc-5a54-4292-b256-6561e66a000d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:15.697046 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-52decbbc-5a54-4292-b256-6561e66a000d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:15.817889 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52decbbc-5a54-4292-b256-6561e66a000d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:15.819702 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52decbbc-5a54-4292-b256-6561e66a000d service nova] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:15.839428 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 708/1406] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:06:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:06:18.115563 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368ce2bb-c287-4960-a94d-443411a7a63a None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:18.116160 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368ce2bb-c287-4960-a94d-443411a7a63a None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:18.116160 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368ce2bb-c287-4960-a94d-443411a7a63a None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:18.119005 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 699/1407] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:06:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:06:18.123492 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c8d377-71e4-4d2d-88cd-88f03d239d18 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:18.123907 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c8d377-71e4-4d2d-88cd-88f03d239d18 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:18.124103 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c8d377-71e4-4d2d-88cd-88f03d239d18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:18.195464 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-98c8d377-71e4-4d2d-88cd-88f03d239d18 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:06:18.196685 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-98c8d377-71e4-4d2d-88cd-88f03d239d18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:18.228525 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:18.309577 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 709/1408] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:06:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:06:18.606180 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368ce2bb-c287-4960-a94d-443411a7a63a None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:18.607910 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-368ce2bb-c287-4960-a94d-443411a7a63a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:18.652370 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:18.781468 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368ce2bb-c287-4960-a94d-443411a7a63a None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:18.905554 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:18.917134 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8b0ab6-f6f3-4658-b013-f4a62e830195 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70bac169d5cd43e1926f310344ac11fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:18.917751 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8b0ab6-f6f3-4658-b013-f4a62e830195 service nova] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:18.918033 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8b0ab6-f6f3-4658-b013-f4a62e830195 service nova] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:18.918303 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8b0ab6-f6f3-4658-b013-f4a62e830195 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:18.919938 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-3d8b0ab6-f6f3-4658-b013-f4a62e830195 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:19.027254 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8b0ab6-f6f3-4658-b013-f4a62e830195 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:19.028959 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8b0ab6-f6f3-4658-b013-f4a62e830195 service nova] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:19.050926 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 700/1409] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:06:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:06:28.308391 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=30 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:29.045360 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=48 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:32.980551 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e2f1b8-ffe8-490b-a9bc-621e5fb70ef9 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:32.981127 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e2f1b8-ffe8-490b-a9bc-621e5fb70ef9 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:32.981127 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e2f1b8-ffe8-490b-a9bc-621e5fb70ef9 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:32.982419 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 710/1410] 10.222.0.49 () {54 vars in 918 bytes} [Fri Sep 9 12:06:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:06:32.987410 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cc41df-bfc2-476e-8322-f4838c39f65d None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:32.987805 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cc41df-bfc2-476e-8322-f4838c39f65d None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:32.987946 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cc41df-bfc2-476e-8322-f4838c39f65d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:33.065777 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-56cc41df-bfc2-476e-8322-f4838c39f65d None None] MFA Rules not processed for user `d3adb2f0583c4ff782049dacdb3ee556`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:06:33.067037 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-56cc41df-bfc2-476e-8322-f4838c39f65d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:33.171368 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 701/1411] 10.222.0.49 () {58 vars in 1015 bytes} [Fri Sep 9 12:06:32 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:06:33.186924 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c3f24d-9c14-4f7c-b529-6791c158f0d3 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:33.187232 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c3f24d-9c14-4f7c-b529-6791c158f0d3 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:33.187507 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c3f24d-9c14-4f7c-b529-6791c158f0d3 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:33.188832 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 711/1412] 10.222.0.49 () {54 vars in 918 bytes} [Fri Sep 9 12:06:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:06:33.193556 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae23137-13d2-42e1-b37c-ac464f3144a4 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:33.193838 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae23137-13d2-42e1-b37c-ac464f3144a4 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:33.194134 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae23137-13d2-42e1-b37c-ac464f3144a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:33.266741 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:33.270200 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-6ae23137-13d2-42e1-b37c-ac464f3144a4 None None] MFA Rules not processed for user `70bac169d5cd43e1926f310344ac11fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:06:33.271988 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-6ae23137-13d2-42e1-b37c-ac464f3144a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:33.388012 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 702/1413] 10.222.0.49 () {58 vars in 1015 bytes} [Fri Sep 9 12:06:33 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:06:35.989615 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dd3413-5624-4253-addd-43da26fbe247 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:35.990456 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dd3413-5624-4253-addd-43da26fbe247 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:35.990742 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dd3413-5624-4253-addd-43da26fbe247 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:35.992182 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 712/1414] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 12:06:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:06:35.998099 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f358bf-e6f8-4e55-9f56-e42b3f561844 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:35.998520 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f358bf-e6f8-4e55-9f56-e42b3f561844 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:35.998623 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f358bf-e6f8-4e55-9f56-e42b3f561844 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:36.064188 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-a2f358bf-e6f8-4e55-9f56-e42b3f561844 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:06:36.065392 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-a2f358bf-e6f8-4e55-9f56-e42b3f561844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:36.119364 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:36.162142 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 703/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 12:06:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:06:36.451102 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dd3413-5624-4253-addd-43da26fbe247 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:36.452683 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-e1dd3413-5624-4253-addd-43da26fbe247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:36.595103 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dd3413-5624-4253-addd-43da26fbe247 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:36.666568 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:36.706421 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45f5f5a-1af6-451b-997c-a01a5b64b2d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70bac169d5cd43e1926f310344ac11fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:36.707063 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45f5f5a-1af6-451b-997c-a01a5b64b2d0 service nova] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:36.707340 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45f5f5a-1af6-451b-997c-a01a5b64b2d0 service nova] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:36.707606 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45f5f5a-1af6-451b-997c-a01a5b64b2d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:36.709186 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-c45f5f5a-1af6-451b-997c-a01a5b64b2d0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:36.796944 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45f5f5a-1af6-451b-997c-a01a5b64b2d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:36.798547 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45f5f5a-1af6-451b-997c-a01a5b64b2d0 service nova] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:36.816686 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 713/1416] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:06:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:06:37.953048 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328f179b-ae3b-48c2-94d1-ae79ee9db8a3 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:37.953574 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328f179b-ae3b-48c2-94d1-ae79ee9db8a3 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:37.953574 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328f179b-ae3b-48c2-94d1-ae79ee9db8a3 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:37.954970 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 704/1417] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 12:06:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:06:37.961040 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d167d844-583b-4452-ae6c-2d0ea6fe0d60 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:37.961806 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d167d844-583b-4452-ae6c-2d0ea6fe0d60 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:37.962109 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d167d844-583b-4452-ae6c-2d0ea6fe0d60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:38.037739 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-d167d844-583b-4452-ae6c-2d0ea6fe0d60 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:06:38.038975 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-d167d844-583b-4452-ae6c-2d0ea6fe0d60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:38.065630 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:38.135082 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 714/1418] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 12:06:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:06:38.289123 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328f179b-ae3b-48c2-94d1-ae79ee9db8a3 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:38.290731 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-328f179b-ae3b-48c2-94d1-ae79ee9db8a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:38.436250 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328f179b-ae3b-48c2-94d1-ae79ee9db8a3 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:38.477133 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:38.552215 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cc997e-a8cb-473a-818c-fa88be11e9a4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70bac169d5cd43e1926f310344ac11fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:38.552751 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cc997e-a8cb-473a-818c-fa88be11e9a4 service nova] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:38.553031 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cc997e-a8cb-473a-818c-fa88be11e9a4 service nova] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:38.553300 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cc997e-a8cb-473a-818c-fa88be11e9a4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:38.555201 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-33cc997e-a8cb-473a-818c-fa88be11e9a4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:38.643740 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cc997e-a8cb-473a-818c-fa88be11e9a4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:38.645360 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cc997e-a8cb-473a-818c-fa88be11e9a4 service nova] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:38.664298 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 705/1419] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:06:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:06:39.801176 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7ab65c-5366-48bc-9c30-9537b1488858 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:39.801584 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7ab65c-5366-48bc-9c30-9537b1488858 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:39.801750 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7ab65c-5366-48bc-9c30-9537b1488858 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:39.803133 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 715/1420] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 12:06:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:06:39.810810 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80ae69a-521e-4800-9d3d-25bbc911e87c None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:39.811342 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80ae69a-521e-4800-9d3d-25bbc911e87c None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:39.811441 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80ae69a-521e-4800-9d3d-25bbc911e87c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:39.865327 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:39.888257 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-a80ae69a-521e-4800-9d3d-25bbc911e87c None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:06:39.889479 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-a80ae69a-521e-4800-9d3d-25bbc911e87c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:39.985503 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 706/1421] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 12:06:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:06:40.139445 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7ab65c-5366-48bc-9c30-9537b1488858 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:40.140973 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-5f7ab65c-5366-48bc-9c30-9537b1488858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:40.285486 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7ab65c-5366-48bc-9c30-9537b1488858 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:40.297977 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:40.398352 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14062911-3314-48b8-852f-af8167d3b28c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70bac169d5cd43e1926f310344ac11fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:40.398983 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14062911-3314-48b8-852f-af8167d3b28c service nova] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:40.399259 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14062911-3314-48b8-852f-af8167d3b28c service nova] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:40.399525 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14062911-3314-48b8-852f-af8167d3b28c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:40.401113 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-14062911-3314-48b8-852f-af8167d3b28c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:40.490337 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14062911-3314-48b8-852f-af8167d3b28c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:40.491986 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14062911-3314-48b8-852f-af8167d3b28c service nova] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:40.510110 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 716/1422] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:06:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:06:41.628324 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfad4ba-8f72-4d64-85d9-2873babf87b7 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:41.628889 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfad4ba-8f72-4d64-85d9-2873babf87b7 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:41.628889 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfad4ba-8f72-4d64-85d9-2873babf87b7 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:41.630223 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 707/1423] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 12:06:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:06:41.636543 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a5ee26-dcc3-4bdb-9038-20befcf13f5b None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:41.636982 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a5ee26-dcc3-4bdb-9038-20befcf13f5b None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:41.637103 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a5ee26-dcc3-4bdb-9038-20befcf13f5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:41.658484 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:41.715197 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-51a5ee26-dcc3-4bdb-9038-20befcf13f5b None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:06:41.717046 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-51a5ee26-dcc3-4bdb-9038-20befcf13f5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:41.812012 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 717/1424] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 12:06:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:06:41.961371 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfad4ba-8f72-4d64-85d9-2873babf87b7 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:41.962813 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-5cfad4ba-8f72-4d64-85d9-2873babf87b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:42.077952 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:42.114474 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfad4ba-8f72-4d64-85d9-2873babf87b7 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:42.226044 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d503c00-0e0c-4304-b6ec-dc726b505fef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70bac169d5cd43e1926f310344ac11fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:42.226561 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d503c00-0e0c-4304-b6ec-dc726b505fef service nova] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:42.226836 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d503c00-0e0c-4304-b6ec-dc726b505fef service nova] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:42.227132 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d503c00-0e0c-4304-b6ec-dc726b505fef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:42.228720 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-2d503c00-0e0c-4304-b6ec-dc726b505fef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:42.303935 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:42.318495 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d503c00-0e0c-4304-b6ec-dc726b505fef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:42.320138 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d503c00-0e0c-4304-b6ec-dc726b505fef service nova] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:42.338467 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 708/1425] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:06:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:06:43.452489 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8017f7-9aa1-4f92-9dbb-94441f240922 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:43.453090 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8017f7-9aa1-4f92-9dbb-94441f240922 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:43.453090 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8017f7-9aa1-4f92-9dbb-94441f240922 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:43.454365 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 718/1426] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 12:06:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:06:43.460471 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0574455b-b611-49ee-95fb-b41d7ad42664 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:43.460937 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0574455b-b611-49ee-95fb-b41d7ad42664 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:43.461043 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0574455b-b611-49ee-95fb-b41d7ad42664 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:43.537410 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-0574455b-b611-49ee-95fb-b41d7ad42664 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:06:43.538611 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-0574455b-b611-49ee-95fb-b41d7ad42664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:43.634020 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 709/1427] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 12:06:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:06:43.783885 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8017f7-9aa1-4f92-9dbb-94441f240922 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:43.785447 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-4d8017f7-9aa1-4f92-9dbb-94441f240922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:43.868218 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:43.930177 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8017f7-9aa1-4f92-9dbb-94441f240922 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:44.042336 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c6d116-2d8a-4ca5-a3ad-bfc94df45470 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70bac169d5cd43e1926f310344ac11fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:44.042945 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c6d116-2d8a-4ca5-a3ad-bfc94df45470 service nova] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:44.043224 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c6d116-2d8a-4ca5-a3ad-bfc94df45470 service nova] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:44.043489 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c6d116-2d8a-4ca5-a3ad-bfc94df45470 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:44.045921 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-78c6d116-2d8a-4ca5-a3ad-bfc94df45470 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:44.094270 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:44.136647 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c6d116-2d8a-4ca5-a3ad-bfc94df45470 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:44.138238 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c6d116-2d8a-4ca5-a3ad-bfc94df45470 service nova] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:44.155593 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 719/1428] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:06:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:06:45.231422 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b212c7b5-77d7-4fe8-9354-a90c1f12822c None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:45.232626 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b212c7b5-77d7-4fe8-9354-a90c1f12822c None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:45.232626 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b212c7b5-77d7-4fe8-9354-a90c1f12822c None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:45.233955 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 710/1429] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 12:06:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:06:45.240213 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977fc898-1caf-42c9-ac1c-fae4bdfe602b None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:45.240623 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977fc898-1caf-42c9-ac1c-fae4bdfe602b None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:45.240758 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977fc898-1caf-42c9-ac1c-fae4bdfe602b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:45.317234 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-977fc898-1caf-42c9-ac1c-fae4bdfe602b None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:06:45.318352 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-977fc898-1caf-42c9-ac1c-fae4bdfe602b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:45.412952 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 720/1430] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 12:06:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:06:45.564452 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b212c7b5-77d7-4fe8-9354-a90c1f12822c None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:45.566025 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-b212c7b5-77d7-4fe8-9354-a90c1f12822c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:45.615121 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:45.712214 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b212c7b5-77d7-4fe8-9354-a90c1f12822c None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:45.823043 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3508057e-8b4d-4c64-afaf-747d0c0076ac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70bac169d5cd43e1926f310344ac11fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:45.823653 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3508057e-8b4d-4c64-afaf-747d0c0076ac service nova] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:45.823935 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3508057e-8b4d-4c64-afaf-747d0c0076ac service nova] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:45.824205 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3508057e-8b4d-4c64-afaf-747d0c0076ac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:45.826440 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-3508057e-8b4d-4c64-afaf-747d0c0076ac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:45.833294 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:45.915319 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3508057e-8b4d-4c64-afaf-747d0c0076ac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:45.916826 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3508057e-8b4d-4c64-afaf-747d0c0076ac service nova] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:45.935811 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 711/1431] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:06:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:06:46.983157 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7111b418-fc3d-4e54-8a1e-c0fff922c5c9 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:46.983736 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7111b418-fc3d-4e54-8a1e-c0fff922c5c9 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:46.983736 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7111b418-fc3d-4e54-8a1e-c0fff922c5c9 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:46.985053 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 721/1432] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 12:06:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:06:46.991361 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da4f6a6-0fd3-4d33-b1a3-9f8b74541ed5 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:46.991796 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da4f6a6-0fd3-4d33-b1a3-9f8b74541ed5 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:46.991909 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da4f6a6-0fd3-4d33-b1a3-9f8b74541ed5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:47.068826 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-2da4f6a6-0fd3-4d33-b1a3-9f8b74541ed5 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:06:47.070045 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-2da4f6a6-0fd3-4d33-b1a3-9f8b74541ed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:47.143527 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:47.173872 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 712/1433] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 12:06:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:06:47.324307 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7111b418-fc3d-4e54-8a1e-c0fff922c5c9 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:47.326505 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-7111b418-fc3d-4e54-8a1e-c0fff922c5c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:47.343723 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:47.471816 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7111b418-fc3d-4e54-8a1e-c0fff922c5c9 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:47.558005 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:47.584346 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20371594-8020-4270-a2b7-e06d0afb0445 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70bac169d5cd43e1926f310344ac11fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:47.584890 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20371594-8020-4270-a2b7-e06d0afb0445 service nova] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:47.585169 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20371594-8020-4270-a2b7-e06d0afb0445 service nova] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:47.585441 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20371594-8020-4270-a2b7-e06d0afb0445 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:47.587067 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-20371594-8020-4270-a2b7-e06d0afb0445 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:47.675754 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20371594-8020-4270-a2b7-e06d0afb0445 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:47.677345 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20371594-8020-4270-a2b7-e06d0afb0445 service nova] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:47.695594 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 722/1434] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:06:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:06:48.753595 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabd3a11-f2e8-4dc9-8922-23620079373d None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:48.754170 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabd3a11-f2e8-4dc9-8922-23620079373d None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:48.754170 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabd3a11-f2e8-4dc9-8922-23620079373d None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:48.755537 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 713/1435] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 12:06:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:06:48.761983 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5330c692-215f-4aaf-a1a0-ef17dbdc17f6 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:48.762482 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5330c692-215f-4aaf-a1a0-ef17dbdc17f6 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:48.762602 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5330c692-215f-4aaf-a1a0-ef17dbdc17f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:48.840122 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-5330c692-215f-4aaf-a1a0-ef17dbdc17f6 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:06:48.841609 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-5330c692-215f-4aaf-a1a0-ef17dbdc17f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:48.882244 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:48.937273 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 723/1436] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 12:06:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:06:49.087855 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabd3a11-f2e8-4dc9-8922-23620079373d None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:49.089361 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-eabd3a11-f2e8-4dc9-8922-23620079373d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:49.241573 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabd3a11-f2e8-4dc9-8922-23620079373d None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:49.295871 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:49.354785 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0415f5ef-fa17-4dee-be89-686a439690b5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70bac169d5cd43e1926f310344ac11fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:49.355416 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0415f5ef-fa17-4dee-be89-686a439690b5 service nova] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:49.355690 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0415f5ef-fa17-4dee-be89-686a439690b5 service nova] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:49.355955 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0415f5ef-fa17-4dee-be89-686a439690b5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:49.357925 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-0415f5ef-fa17-4dee-be89-686a439690b5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:49.445797 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0415f5ef-fa17-4dee-be89-686a439690b5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:49.447421 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0415f5ef-fa17-4dee-be89-686a439690b5 service nova] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:49.465699 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 714/1437] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:06:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:06:50.532038 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6a316e-49fd-4667-a0d1-6b86b6376124 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:50.532628 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6a316e-49fd-4667-a0d1-6b86b6376124 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:50.532628 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6a316e-49fd-4667-a0d1-6b86b6376124 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:50.533910 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 724/1438] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 12:06:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:06:50.540409 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54459c2c-097a-43ca-a8c6-167c22abe48d None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:50.540840 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54459c2c-097a-43ca-a8c6-167c22abe48d None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:50.540943 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54459c2c-097a-43ca-a8c6-167c22abe48d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:50.617496 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-54459c2c-097a-43ca-a8c6-167c22abe48d None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:06:50.618688 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-54459c2c-097a-43ca-a8c6-167c22abe48d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:50.620677 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:50.715414 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 715/1439] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 12:06:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:06:50.866223 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6a316e-49fd-4667-a0d1-6b86b6376124 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:50.867777 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-ae6a316e-49fd-4667-a0d1-6b86b6376124 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:51.012063 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6a316e-49fd-4667-a0d1-6b86b6376124 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:51.038588 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:51.123731 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f84ee43-7aef-4210-9bb9-4031e391df74 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70bac169d5cd43e1926f310344ac11fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:51.124328 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f84ee43-7aef-4210-9bb9-4031e391df74 service nova] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:51.124606 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f84ee43-7aef-4210-9bb9-4031e391df74 service nova] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:51.124874 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f84ee43-7aef-4210-9bb9-4031e391df74 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:51.126473 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-0f84ee43-7aef-4210-9bb9-4031e391df74 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:51.219344 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f84ee43-7aef-4210-9bb9-4031e391df74 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:51.220947 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f84ee43-7aef-4210-9bb9-4031e391df74 service nova] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:51.238609 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 725/1440] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:06:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:06:53.303657 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac18250-f487-4726-9a41-f69580bdb44e None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:53.304238 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac18250-f487-4726-9a41-f69580bdb44e None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:53.304238 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac18250-f487-4726-9a41-f69580bdb44e None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:53.305864 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 716/1441] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 12:06:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:06:53.311591 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0399177c-09ea-40cf-8dc1-53106fa7d489 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:53.312007 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0399177c-09ea-40cf-8dc1-53106fa7d489 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:53.312134 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0399177c-09ea-40cf-8dc1-53106fa7d489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:53.353277 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:53.378608 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-0399177c-09ea-40cf-8dc1-53106fa7d489 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:06:53.379847 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-0399177c-09ea-40cf-8dc1-53106fa7d489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:53.476347 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 726/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 12:06:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:06:53.767144 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac18250-f487-4726-9a41-f69580bdb44e None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:53.768691 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-fac18250-f487-4726-9a41-f69580bdb44e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:53.871013 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac18250-f487-4726-9a41-f69580bdb44e None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:53.903268 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:53.946796 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b123c9f-0679-4f24-a4af-270a83f0123c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:53.947440 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b123c9f-0679-4f24-a4af-270a83f0123c service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:53.947716 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b123c9f-0679-4f24-a4af-270a83f0123c service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:53.947986 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b123c9f-0679-4f24-a4af-270a83f0123c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:53.949600 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-8b123c9f-0679-4f24-a4af-270a83f0123c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:54.058530 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b123c9f-0679-4f24-a4af-270a83f0123c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:54.060437 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b123c9f-0679-4f24-a4af-270a83f0123c service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:54.079988 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 717/1443] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:06:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:06:56.448967 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d015388b-03c6-443c-82e7-b3732e6bf1f6 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:56.449433 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d015388b-03c6-443c-82e7-b3732e6bf1f6 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:56.449545 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d015388b-03c6-443c-82e7-b3732e6bf1f6 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:56.451488 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 727/1444] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 12:06:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:06:56.456768 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f938b8-47f6-42da-ae04-54ea91eecac7 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:56.457197 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f938b8-47f6-42da-ae04-54ea91eecac7 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:56.457297 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f938b8-47f6-42da-ae04-54ea91eecac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:56.522237 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-58f938b8-47f6-42da-ae04-54ea91eecac7 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:06:56.523455 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-58f938b8-47f6-42da-ae04-54ea91eecac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:56.535941 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:56.619707 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 718/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 12:06:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:06:56.901161 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d015388b-03c6-443c-82e7-b3732e6bf1f6 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:06:56.902675 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-d015388b-03c6-443c-82e7-b3732e6bf1f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:57.006977 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d015388b-03c6-443c-82e7-b3732e6bf1f6 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:06:57.024674 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:57.080760 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310de4d0-c294-45a1-92d6-f8435ac19731 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:06:57.081359 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310de4d0-c294-45a1-92d6-f8435ac19731 service neutron] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:57.081629 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310de4d0-c294-45a1-92d6-f8435ac19731 service neutron] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:57.081894 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310de4d0-c294-45a1-92d6-f8435ac19731 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:57.083490 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-310de4d0-c294-45a1-92d6-f8435ac19731 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:57.195433 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310de4d0-c294-45a1-92d6-f8435ac19731 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:06:57.196777 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310de4d0-c294-45a1-92d6-f8435ac19731 service neutron] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:06:57.214289 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 728/1446] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:06:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:06:59.701444 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a13602-fdb4-4beb-a16c-5bbc3097d59d None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:59.702002 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a13602-fdb4-4beb-a16c-5bbc3097d59d None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:59.702002 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a13602-fdb4-4beb-a16c-5bbc3097d59d None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:59.703355 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 719/1447] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 12:06:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:06:59.708704 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85874f7-c3a8-4aab-bdbc-7ff6941ac7df None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:06:59.709102 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85874f7-c3a8-4aab-bdbc-7ff6941ac7df None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:06:59.709251 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85874f7-c3a8-4aab-bdbc-7ff6941ac7df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:06:59.758480 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:06:59.776408 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-d85874f7-c3a8-4aab-bdbc-7ff6941ac7df None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:06:59.777622 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-d85874f7-c3a8-4aab-bdbc-7ff6941ac7df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:06:59.873630 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 729/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 12:06:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:07:00.158130 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a13602-fdb4-4beb-a16c-5bbc3097d59d None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:00.159709 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-20a13602-fdb4-4beb-a16c-5bbc3097d59d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:00.262455 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a13602-fdb4-4beb-a16c-5bbc3097d59d None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:00.302297 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:07:00.337658 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e0d788-280b-4b67-b906-a72ef93c8e85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:00.338262 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e0d788-280b-4b67-b906-a72ef93c8e85 service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:00.338540 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e0d788-280b-4b67-b906-a72ef93c8e85 service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:00.338811 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e0d788-280b-4b67-b906-a72ef93c8e85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:00.340431 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-33e0d788-280b-4b67-b906-a72ef93c8e85 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:00.453269 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e0d788-280b-4b67-b906-a72ef93c8e85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:00.454694 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e0d788-280b-4b67-b906-a72ef93c8e85 service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:00.472650 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 720/1449] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:07:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:07:02.760182 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ced3bd9-830d-437a-bf9b-3f055c64b19e None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:02.760707 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ced3bd9-830d-437a-bf9b-3f055c64b19e None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:02.760707 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ced3bd9-830d-437a-bf9b-3f055c64b19e None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:02.762079 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 730/1450] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 12:07:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:07:02.767723 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adf946e-74f8-4806-9f3f-9e8e2750b1f4 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:02.768137 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adf946e-74f8-4806-9f3f-9e8e2750b1f4 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:02.768269 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adf946e-74f8-4806-9f3f-9e8e2750b1f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:02.835032 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-5adf946e-74f8-4806-9f3f-9e8e2750b1f4 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:07:02.836238 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-5adf946e-74f8-4806-9f3f-9e8e2750b1f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:02.856221 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:07:02.933136 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 721/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 12:07:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:07:03.220557 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ced3bd9-830d-437a-bf9b-3f055c64b19e None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:03.222095 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-9ced3bd9-830d-437a-bf9b-3f055c64b19e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:03.323921 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ced3bd9-830d-437a-bf9b-3f055c64b19e None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:03.344952 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:07:03.397433 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642aae36-05ad-47a1-b275-561339e220f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:03.398030 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642aae36-05ad-47a1-b275-561339e220f6 service neutron] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:03.398303 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642aae36-05ad-47a1-b275-561339e220f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:03.398569 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642aae36-05ad-47a1-b275-561339e220f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:03.400167 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-642aae36-05ad-47a1-b275-561339e220f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:03.507290 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642aae36-05ad-47a1-b275-561339e220f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:03.508792 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642aae36-05ad-47a1-b275-561339e220f6 service neutron] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:03.526644 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 731/1452] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:07:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:07:05.725686 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54950dea-3939-4233-801d-d3f72a7d756a None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:05.726265 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54950dea-3939-4233-801d-d3f72a7d756a None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:05.726265 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54950dea-3939-4233-801d-d3f72a7d756a None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:05.727622 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 722/1453] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 12:07:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:07:05.733873 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfc9c67-dd26-46f8-b5fd-a770dc83560f None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:05.734277 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfc9c67-dd26-46f8-b5fd-a770dc83560f None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:05.734410 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfc9c67-dd26-46f8-b5fd-a770dc83560f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:05.797944 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:07:05.800715 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-fbfc9c67-dd26-46f8-b5fd-a770dc83560f None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:07:05.802547 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-fbfc9c67-dd26-46f8-b5fd-a770dc83560f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:05.898742 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 732/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 12:07:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:07:06.183456 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54950dea-3939-4233-801d-d3f72a7d756a None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:06.185551 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-54950dea-3939-4233-801d-d3f72a7d756a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:06.292862 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54950dea-3939-4233-801d-d3f72a7d756a None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:06.342725 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:07:06.366724 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aec8976-3853-4166-8958-8ce7285ed350 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:06.367345 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aec8976-3853-4166-8958-8ce7285ed350 service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:06.367618 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aec8976-3853-4166-8958-8ce7285ed350 service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:06.367879 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aec8976-3853-4166-8958-8ce7285ed350 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:06.369480 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-2aec8976-3853-4166-8958-8ce7285ed350 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:06.478155 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aec8976-3853-4166-8958-8ce7285ed350 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:06.481123 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aec8976-3853-4166-8958-8ce7285ed350 service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:06.499567 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 723/1455] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:07:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:07:08.725415 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232c68a6-e900-43af-8f47-2c7ff843ecfe None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:08.725916 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232c68a6-e900-43af-8f47-2c7ff843ecfe None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:08.725916 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232c68a6-e900-43af-8f47-2c7ff843ecfe None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:08.727243 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 733/1456] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 12:07:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:07:08.732718 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6ed339-67ca-4c61-9c15-6d6a0e27fc59 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:08.733121 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6ed339-67ca-4c61-9c15-6d6a0e27fc59 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:08.733276 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6ed339-67ca-4c61-9c15-6d6a0e27fc59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:08.801510 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-2e6ed339-67ca-4c61-9c15-6d6a0e27fc59 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:07:08.802721 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-2e6ed339-67ca-4c61-9c15-6d6a0e27fc59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:08.831780 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:07:08.898827 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 724/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 12:07:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:07:09.181010 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232c68a6-e900-43af-8f47-2c7ff843ecfe None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:09.182870 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-232c68a6-e900-43af-8f47-2c7ff843ecfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:09.286179 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232c68a6-e900-43af-8f47-2c7ff843ecfe None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:09.317265 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:07:09.363130 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d3cce2-6e9b-417f-8d0f-35e069def2f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:09.363728 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d3cce2-6e9b-417f-8d0f-35e069def2f3 service neutron] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:09.364000 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d3cce2-6e9b-417f-8d0f-35e069def2f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:09.364269 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d3cce2-6e9b-417f-8d0f-35e069def2f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:09.365848 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-34d3cce2-6e9b-417f-8d0f-35e069def2f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:09.472994 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d3cce2-6e9b-417f-8d0f-35e069def2f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:09.474365 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d3cce2-6e9b-417f-8d0f-35e069def2f3 service neutron] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:09.492214 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 734/1458] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:07:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:07:11.743458 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e27a809-8656-40ea-bac4-8b8c2b945310 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:11.744033 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e27a809-8656-40ea-bac4-8b8c2b945310 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:11.744033 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e27a809-8656-40ea-bac4-8b8c2b945310 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:11.745335 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 725/1459] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 12:07:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:07:11.751631 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4be54d5-d4f1-49b7-bd4f-c95b15764aae None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:11.752066 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4be54d5-d4f1-49b7-bd4f-c95b15764aae None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:11.752180 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4be54d5-d4f1-49b7-bd4f-c95b15764aae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:11.817180 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-b4be54d5-d4f1-49b7-bd4f-c95b15764aae None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:07:11.818393 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-b4be54d5-d4f1-49b7-bd4f-c95b15764aae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:11.826619 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:07:11.914325 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 735/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 12:07:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:07:12.200607 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e27a809-8656-40ea-bac4-8b8c2b945310 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:12.202444 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-1e27a809-8656-40ea-bac4-8b8c2b945310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:12.305155 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e27a809-8656-40ea-bac4-8b8c2b945310 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:12.360564 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:07:12.378412 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e55906-d4d6-4058-9afc-68b3e7eb54ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:12.379049 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e55906-d4d6-4058-9afc-68b3e7eb54ca service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:12.379323 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e55906-d4d6-4058-9afc-68b3e7eb54ca service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:12.379587 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e55906-d4d6-4058-9afc-68b3e7eb54ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:12.381154 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-50e55906-d4d6-4058-9afc-68b3e7eb54ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:12.490108 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e55906-d4d6-4058-9afc-68b3e7eb54ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:12.491517 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e55906-d4d6-4058-9afc-68b3e7eb54ca service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:12.509105 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 726/1461] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:07:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:07:14.731048 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ea3655-2b14-4b43-80fc-2ddb20712c59 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:14.731653 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ea3655-2b14-4b43-80fc-2ddb20712c59 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:14.731653 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ea3655-2b14-4b43-80fc-2ddb20712c59 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:14.733588 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 736/1462] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 12:07:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:07:14.738906 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c88b9d-4a24-4055-8cfe-6b53439d45f2 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:14.739317 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c88b9d-4a24-4055-8cfe-6b53439d45f2 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:14.739464 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c88b9d-4a24-4055-8cfe-6b53439d45f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:14.804561 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-b8c88b9d-4a24-4055-8cfe-6b53439d45f2 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:07:14.805773 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-b8c88b9d-4a24-4055-8cfe-6b53439d45f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:14.845343 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:07:14.903393 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 727/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 12:07:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:07:15.187775 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ea3655-2b14-4b43-80fc-2ddb20712c59 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:15.189291 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-40ea3655-2b14-4b43-80fc-2ddb20712c59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:15.290868 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ea3655-2b14-4b43-80fc-2ddb20712c59 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:15.333288 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:07:15.369307 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cd52da-be92-4b93-81e7-520b54eb5ee3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:15.369909 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cd52da-be92-4b93-81e7-520b54eb5ee3 service neutron] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:15.370187 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cd52da-be92-4b93-81e7-520b54eb5ee3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:15.370454 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cd52da-be92-4b93-81e7-520b54eb5ee3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:15.372108 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-47cd52da-be92-4b93-81e7-520b54eb5ee3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:15.478795 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cd52da-be92-4b93-81e7-520b54eb5ee3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:15.480172 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cd52da-be92-4b93-81e7-520b54eb5ee3 service neutron] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:15.497682 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 737/1464] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:07:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:07:17.743318 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ae4b4e-7a09-4e9d-8f90-adb4969d3dfc None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:17.743882 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ae4b4e-7a09-4e9d-8f90-adb4969d3dfc None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:17.743882 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ae4b4e-7a09-4e9d-8f90-adb4969d3dfc None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:17.745215 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 728/1465] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 12:07:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:07:17.750477 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c56484c-fc97-4aa4-8619-f4d87f934864 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:17.750946 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c56484c-fc97-4aa4-8619-f4d87f934864 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:17.751095 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c56484c-fc97-4aa4-8619-f4d87f934864 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:17.816947 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-0c56484c-fc97-4aa4-8619-f4d87f934864 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:07:17.818157 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-0c56484c-fc97-4aa4-8619-f4d87f934864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:17.836665 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:07:17.914847 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 738/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 12:07:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:07:18.195019 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ae4b4e-7a09-4e9d-8f90-adb4969d3dfc None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:18.196579 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-b0ae4b4e-7a09-4e9d-8f90-adb4969d3dfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:18.299244 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ae4b4e-7a09-4e9d-8f90-adb4969d3dfc None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:18.366755 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:07:18.373967 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf39fc1-83d9-4915-a9df-92436fc4e285 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:18.374579 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf39fc1-83d9-4915-a9df-92436fc4e285 service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:18.374856 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf39fc1-83d9-4915-a9df-92436fc4e285 service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:18.375152 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf39fc1-83d9-4915-a9df-92436fc4e285 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:18.376760 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-dcf39fc1-83d9-4915-a9df-92436fc4e285 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:18.487141 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf39fc1-83d9-4915-a9df-92436fc4e285 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:18.488487 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf39fc1-83d9-4915-a9df-92436fc4e285 service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:18.506072 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 729/1467] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:07:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:07:20.664787 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01b3ba7-be78-4e14-819c-d5333045818f None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:20.665217 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01b3ba7-be78-4e14-819c-d5333045818f None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:20.665331 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01b3ba7-be78-4e14-819c-d5333045818f None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:20.666695 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 739/1468] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 12:07:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:07:20.672737 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba383b6-a791-41b0-a556-a71e1ef2ea9b None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:20.673185 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba383b6-a791-41b0-a556-a71e1ef2ea9b None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:20.673292 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba383b6-a791-41b0-a556-a71e1ef2ea9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:20.739573 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-cba383b6-a791-41b0-a556-a71e1ef2ea9b None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:07:20.740716 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-cba383b6-a791-41b0-a556-a71e1ef2ea9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:20.789930 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:07:20.840256 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 730/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 12:07:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:07:21.127570 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01b3ba7-be78-4e14-819c-d5333045818f None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:21.129133 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-a01b3ba7-be78-4e14-819c-d5333045818f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:21.232563 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01b3ba7-be78-4e14-819c-d5333045818f None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:21.409736 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:07:21.440437 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9514b86-2c74-48c4-a300-6bc8ae7371af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:21.441040 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9514b86-2c74-48c4-a300-6bc8ae7371af service neutron] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:21.441311 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9514b86-2c74-48c4-a300-6bc8ae7371af service neutron] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:21.441573 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9514b86-2c74-48c4-a300-6bc8ae7371af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:21.443208 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-a9514b86-2c74-48c4-a300-6bc8ae7371af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:21.572805 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9514b86-2c74-48c4-a300-6bc8ae7371af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:21.574277 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9514b86-2c74-48c4-a300-6bc8ae7371af service neutron] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:21.595111 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 740/1470] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:07:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:07:30.838744 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=22 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:07:31.592671 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=66 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:07:41.809785 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b38300e-3529-42e8-8476-4022dd89717b None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:41.810387 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b38300e-3529-42e8-8476-4022dd89717b None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:41.810485 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b38300e-3529-42e8-8476-4022dd89717b None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:41.812099 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 731/1471] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:07:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:07:41.817554 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e761a3e6-495b-4b3a-a917-ea712a37da9b None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:41.818006 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e761a3e6-495b-4b3a-a917-ea712a37da9b None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:41.818105 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e761a3e6-495b-4b3a-a917-ea712a37da9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:41.882411 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-e761a3e6-495b-4b3a-a917-ea712a37da9b None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:07:41.883643 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-e761a3e6-495b-4b3a-a917-ea712a37da9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:41.979305 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 741/1472] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:07:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:07:42.285218 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6f3ab3-d9aa-4b4f-9278-246b67509428 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:42.285483 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6f3ab3-d9aa-4b4f-9278-246b67509428 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:42.285748 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6f3ab3-d9aa-4b4f-9278-246b67509428 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:42.287199 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 732/1473] 10.222.0.49 () {54 vars in 966 bytes} [Fri Sep 9 12:07:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:07:42.292082 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534b2202-ba10-48f3-a410-57dddcd7edfb None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:42.292388 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534b2202-ba10-48f3-a410-57dddcd7edfb None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:42.292657 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534b2202-ba10-48f3-a410-57dddcd7edfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:42.368585 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-534b2202-ba10-48f3-a410-57dddcd7edfb None None] MFA Rules not processed for user `37c4e14aa8e04d3e85a406b9363bca2a`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:07:42.369778 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-534b2202-ba10-48f3-a410-57dddcd7edfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:42.392439 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:07:42.447317 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 742/1474] 10.222.0.49 () {58 vars in 1063 bytes} [Fri Sep 9 12:07:42 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:07:42.455667 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6f3ab3-d9aa-4b4f-9278-246b67509428 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:42.457175 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-6a6f3ab3-d9aa-4b4f-9278-246b67509428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:42.681963 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6f3ab3-d9aa-4b4f-9278-246b67509428 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:42.756285 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f955c4b8-28c7-474e-91a2-1493622cf747 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37c4e14aa8e04d3e85a406b9363bca2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:42.756889 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f955c4b8-28c7-474e-91a2-1493622cf747 service glance] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:42.757162 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f955c4b8-28c7-474e-91a2-1493622cf747 service glance] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:42.757428 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f955c4b8-28c7-474e-91a2-1493622cf747 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:42.759062 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-f955c4b8-28c7-474e-91a2-1493622cf747 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:42.815675 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:07:42.872038 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f955c4b8-28c7-474e-91a2-1493622cf747 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:42.874021 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f955c4b8-28c7-474e-91a2-1493622cf747 service glance] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:42.892424 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 733/1475] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:07:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:07:45.153181 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b888d600-9c53-4034-8fe0-4b1966e8b71e None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:45.153811 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b888d600-9c53-4034-8fe0-4b1966e8b71e None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:45.153811 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b888d600-9c53-4034-8fe0-4b1966e8b71e None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:45.155637 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 743/1476] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:07:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:07:45.161303 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb571713-72e6-4773-aa1b-94431c1bdc2b None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:45.161746 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb571713-72e6-4773-aa1b-94431c1bdc2b None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:45.161851 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb571713-72e6-4773-aa1b-94431c1bdc2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:45.227447 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-eb571713-72e6-4773-aa1b-94431c1bdc2b None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:07:45.228674 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-eb571713-72e6-4773-aa1b-94431c1bdc2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:45.324560 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 734/1477] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:07:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:07:45.603682 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b888d600-9c53-4034-8fe0-4b1966e8b71e None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:45.605206 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-b888d600-9c53-4034-8fe0-4b1966e8b71e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:45.728593 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b888d600-9c53-4034-8fe0-4b1966e8b71e None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:45.777104 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:07:45.820595 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457819f6-c57f-4838-862c-c96e0bbb9e74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:45.821190 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457819f6-c57f-4838-862c-c96e0bbb9e74 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:45.821460 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457819f6-c57f-4838-862c-c96e0bbb9e74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:45.821720 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457819f6-c57f-4838-862c-c96e0bbb9e74 admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:45.822341 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457819f6-c57f-4838-862c-c96e0bbb9e74 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:45.824074 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: WARNING py.warnings [None req-457819f6-c57f-4838-862c-c96e0bbb9e74 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 12:07:45.824074 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: warnings.warn(msg) Sep 09 12:07:45.824074 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: Sep 09 12:07:45.825199 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457819f6-c57f-4838-862c-c96e0bbb9e74 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:45.844634 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 744/1478] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 9 12:07:45 2022] GET /identity/v3/projects => generated 2553 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:07:48.011186 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad339ef0-984c-4034-b0de-68f4484f9470 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:48.011758 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad339ef0-984c-4034-b0de-68f4484f9470 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:48.011758 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad339ef0-984c-4034-b0de-68f4484f9470 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:48.013133 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 735/1479] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:07:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:07:48.018554 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2fbd79-114a-4eb3-b284-5c2bc05fc89c None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:48.019025 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2fbd79-114a-4eb3-b284-5c2bc05fc89c None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:48.019149 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2fbd79-114a-4eb3-b284-5c2bc05fc89c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:48.083446 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-aa2fbd79-114a-4eb3-b284-5c2bc05fc89c None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:07:48.084641 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-aa2fbd79-114a-4eb3-b284-5c2bc05fc89c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:48.182234 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 745/1480] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:07:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:07:48.475739 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad339ef0-984c-4034-b0de-68f4484f9470 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:48.477346 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-ad339ef0-984c-4034-b0de-68f4484f9470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:48.487404 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:07:48.620802 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad339ef0-984c-4034-b0de-68f4484f9470 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:48.700533 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:07:48.733783 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43d7ac3-ca02-43d2-8e0d-89632153a4c2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70bac169d5cd43e1926f310344ac11fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:48.734503 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43d7ac3-ca02-43d2-8e0d-89632153a4c2 service nova] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:48.734784 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43d7ac3-ca02-43d2-8e0d-89632153a4c2 service nova] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:48.735085 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43d7ac3-ca02-43d2-8e0d-89632153a4c2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:48.736676 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-f43d7ac3-ca02-43d2-8e0d-89632153a4c2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:48.826060 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43d7ac3-ca02-43d2-8e0d-89632153a4c2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:48.828286 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43d7ac3-ca02-43d2-8e0d-89632153a4c2 service nova] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:48.846993 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 736/1481] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:07:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:07:51.198272 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59000c8-eaf9-44d3-b15c-f4b442865e3e None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:51.198877 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59000c8-eaf9-44d3-b15c-f4b442865e3e None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:51.198877 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59000c8-eaf9-44d3-b15c-f4b442865e3e None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:51.200193 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 746/1482] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:07:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:07:51.205684 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d622b29-67b8-408a-82b8-94e20bc17dac None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:51.206087 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d622b29-67b8-408a-82b8-94e20bc17dac None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:51.206338 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d622b29-67b8-408a-82b8-94e20bc17dac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:51.271631 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-7d622b29-67b8-408a-82b8-94e20bc17dac None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:07:51.272859 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-7d622b29-67b8-408a-82b8-94e20bc17dac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:51.319691 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:07:51.370297 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 737/1483] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:07:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:07:51.664010 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59000c8-eaf9-44d3-b15c-f4b442865e3e None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:51.665538 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-c59000c8-eaf9-44d3-b15c-f4b442865e3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:51.688596 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:07:51.770431 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59000c8-eaf9-44d3-b15c-f4b442865e3e None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:51.843082 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70873813-da00-4e67-a9b9-92961b798327 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:51.843686 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70873813-da00-4e67-a9b9-92961b798327 service neutron] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:51.843965 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70873813-da00-4e67-a9b9-92961b798327 service neutron] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:51.844233 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70873813-da00-4e67-a9b9-92961b798327 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:51.845822 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-70873813-da00-4e67-a9b9-92961b798327 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:51.917942 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:07:51.953576 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70873813-da00-4e67-a9b9-92961b798327 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:51.955033 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70873813-da00-4e67-a9b9-92961b798327 service neutron] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:51.972787 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 747/1484] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:07:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:07:54.181929 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f21d58e-6fa2-468a-96ab-604eb9eca957 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:54.182497 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f21d58e-6fa2-468a-96ab-604eb9eca957 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:54.182497 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f21d58e-6fa2-468a-96ab-604eb9eca957 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:54.184121 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 738/1485] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:07:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:07:54.189504 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fb1009-c108-4c8f-aec4-ce2b363d3f8d None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:54.189936 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fb1009-c108-4c8f-aec4-ce2b363d3f8d None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:54.190040 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fb1009-c108-4c8f-aec4-ce2b363d3f8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:54.255226 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-57fb1009-c108-4c8f-aec4-ce2b363d3f8d None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:07:54.256437 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-57fb1009-c108-4c8f-aec4-ce2b363d3f8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:54.354168 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 748/1486] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:07:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:07:54.642651 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f21d58e-6fa2-468a-96ab-604eb9eca957 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:54.644211 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-4f21d58e-6fa2-468a-96ab-604eb9eca957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:54.747613 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f21d58e-6fa2-468a-96ab-604eb9eca957 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:54.819299 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:07:54.820828 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613d8422-ed9c-49d8-b456-78afb64ddd16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:54.821381 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613d8422-ed9c-49d8-b456-78afb64ddd16 service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:54.821883 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613d8422-ed9c-49d8-b456-78afb64ddd16 service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:54.822336 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613d8422-ed9c-49d8-b456-78afb64ddd16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:54.824179 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-613d8422-ed9c-49d8-b456-78afb64ddd16 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:54.935331 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613d8422-ed9c-49d8-b456-78afb64ddd16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:54.936764 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613d8422-ed9c-49d8-b456-78afb64ddd16 service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:54.954265 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 739/1487] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:07:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:07:57.254561 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b363e63a-54f5-45b0-b4e4-ac14f3d050e3 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:57.255134 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b363e63a-54f5-45b0-b4e4-ac14f3d050e3 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:57.255134 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b363e63a-54f5-45b0-b4e4-ac14f3d050e3 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:57.256478 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 749/1488] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:07:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:07:57.261798 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cbc809-6ffe-46f2-a8d6-071a71e15d6b None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:57.262239 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cbc809-6ffe-46f2-a8d6-071a71e15d6b None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:57.262351 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cbc809-6ffe-46f2-a8d6-071a71e15d6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:57.328485 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-59cbc809-6ffe-46f2-a8d6-071a71e15d6b None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:07:57.329677 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-59cbc809-6ffe-46f2-a8d6-071a71e15d6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:57.381333 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:07:57.427237 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 740/1489] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:07:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:07:57.718430 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b363e63a-54f5-45b0-b4e4-ac14f3d050e3 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:07:57.720088 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-b363e63a-54f5-45b0-b4e4-ac14f3d050e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:57.756658 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:07:57.824155 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b363e63a-54f5-45b0-b4e4-ac14f3d050e3 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:07:57.896740 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4309b6c6-3709-4bb4-b0a0-73f5c157022d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:07:57.897340 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4309b6c6-3709-4bb4-b0a0-73f5c157022d service neutron] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:07:57.897544 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4309b6c6-3709-4bb4-b0a0-73f5c157022d service neutron] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:07:57.897810 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4309b6c6-3709-4bb4-b0a0-73f5c157022d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:07:57.899411 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-4309b6c6-3709-4bb4-b0a0-73f5c157022d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:07:57.977576 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:07:58.007132 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4309b6c6-3709-4bb4-b0a0-73f5c157022d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:07:58.008534 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4309b6c6-3709-4bb4-b0a0-73f5c157022d service neutron] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:07:58.026542 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 750/1490] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:07:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:08:00.419572 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5ed9a6-01a5-4687-b602-21bf8ee4712c None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:08:00.420116 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5ed9a6-01a5-4687-b602-21bf8ee4712c None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:08:00.420116 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5ed9a6-01a5-4687-b602-21bf8ee4712c None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:08:00.421700 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 741/1491] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 9 12:08:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:08:00.426828 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d1c733-0c65-4e50-a702-555fc5962589 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:08:00.427255 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d1c733-0c65-4e50-a702-555fc5962589 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:08:00.427410 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d1c733-0c65-4e50-a702-555fc5962589 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:08:00.493111 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-72d1c733-0c65-4e50-a702-555fc5962589 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:08:00.494300 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-72d1c733-0c65-4e50-a702-555fc5962589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:08:00.588585 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 751/1492] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 9 12:08:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:08:00.883717 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5ed9a6-01a5-4687-b602-21bf8ee4712c None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:08:00.885317 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-ae5ed9a6-01a5-4687-b602-21bf8ee4712c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:08:00.988109 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5ed9a6-01a5-4687-b602-21bf8ee4712c None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:08:01.062464 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e52ba7d-81c6-41bf-8f64-f817c9b0c1f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:08:01.063085 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e52ba7d-81c6-41bf-8f64-f817c9b0c1f4 service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:08:01.063366 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e52ba7d-81c6-41bf-8f64-f817c9b0c1f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:08:01.063637 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e52ba7d-81c6-41bf-8f64-f817c9b0c1f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:08:01.065210 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-9e52ba7d-81c6-41bf-8f64-f817c9b0c1f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:08:01.074603 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:08:01.176016 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e52ba7d-81c6-41bf-8f64-f817c9b0c1f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:08:01.177369 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e52ba7d-81c6-41bf-8f64-f817c9b0c1f4 service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:08:01.195287 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 742/1493] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:08:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:08:10.587096 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=82 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:08:11.193095 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=50 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:09:14.078516 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d61a0ed-f4d4-49ad-8059-1e4193be4fda None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:14.079151 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d61a0ed-f4d4-49ad-8059-1e4193be4fda None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:14.079151 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d61a0ed-f4d4-49ad-8059-1e4193be4fda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:14.160774 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-5d61a0ed-f4d4-49ad-8059-1e4193be4fda None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:09:14.161964 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-5d61a0ed-f4d4-49ad-8059-1e4193be4fda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:09:14.255802 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 752/1494] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:09:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:09:14.264557 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27102b8f-56f8-4a34-b715-1e5d4124f8db None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:14.265097 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27102b8f-56f8-4a34-b715-1e5d4124f8db None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:14.265510 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27102b8f-56f8-4a34-b715-1e5d4124f8db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:14.344023 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-27102b8f-56f8-4a34-b715-1e5d4124f8db None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:09:14.345150 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-27102b8f-56f8-4a34-b715-1e5d4124f8db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:09:14.439644 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 743/1495] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:09:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:09:14.444210 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d61a0ed-f4d4-49ad-8059-1e4193be4fda None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:09:14.445396 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-5d61a0ed-f4d4-49ad-8059-1e4193be4fda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:09:14.457116 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d61a0ed-f4d4-49ad-8059-1e4193be4fda None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:09:14.517237 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:09:14.550468 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20ed256-cb29-4a63-ac73-5afcc6d3caea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:09:14.551097 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20ed256-cb29-4a63-ac73-5afcc6d3caea admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:14.551374 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20ed256-cb29-4a63-ac73-5afcc6d3caea admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:14.551641 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20ed256-cb29-4a63-ac73-5afcc6d3caea admin admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:14.552275 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20ed256-cb29-4a63-ac73-5afcc6d3caea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:09:14.552571 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20ed256-cb29-4a63-ac73-5afcc6d3caea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:09:14.553599 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: WARNING py.warnings [None req-b20ed256-cb29-4a63-ac73-5afcc6d3caea admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 09 12:09:14.553599 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: warnings.warn(msg) Sep 09 12:09:14.553599 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: Sep 09 12:09:14.554482 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20ed256-cb29-4a63-ac73-5afcc6d3caea admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:09:14.566481 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 753/1496] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 9 12:09:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:09:14.574670 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27102b8f-56f8-4a34-b715-1e5d4124f8db None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:09:14.584482 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27102b8f-56f8-4a34-b715-1e5d4124f8db None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:09:14.646709 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:09:14.683367 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d23b16-decf-47d0-92f5-166d2fd1e2b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:09:14.684009 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d23b16-decf-47d0-92f5-166d2fd1e2b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:14.684285 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d23b16-decf-47d0-92f5-166d2fd1e2b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:14.684548 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d23b16-decf-47d0-92f5-166d2fd1e2b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:14.685182 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d23b16-decf-47d0-92f5-166d2fd1e2b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:09:14.687875 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: WARNING py.warnings [None req-28d23b16-decf-47d0-92f5-166d2fd1e2b7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 12:09:14.687875 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: warnings.warn(msg) Sep 09 12:09:14.687875 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: Sep 09 12:09:14.689140 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d23b16-decf-47d0-92f5-166d2fd1e2b7 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:09:14.745933 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 744/1497] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 9 12:09:14 2022] POST /identity/v3/projects => generated 375 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:09:14.750820 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20ed256-cb29-4a63-ac73-5afcc6d3caea admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:09:14.752434 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-b20ed256-cb29-4a63-ac73-5afcc6d3caea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:09:14.785540 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20ed256-cb29-4a63-ac73-5afcc6d3caea admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:09:14.877980 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618c796e-d820-439e-a9c5-6f3eb2e5c952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:09:14.878575 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618c796e-d820-439e-a9c5-6f3eb2e5c952 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:14.878846 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618c796e-d820-439e-a9c5-6f3eb2e5c952 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:14.879139 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618c796e-d820-439e-a9c5-6f3eb2e5c952 admin admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:14.879761 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618c796e-d820-439e-a9c5-6f3eb2e5c952 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:09:14.880753 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: WARNING py.warnings [None req-618c796e-d820-439e-a9c5-6f3eb2e5c952 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 12:09:14.880753 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: warnings.warn(msg) Sep 09 12:09:14.880753 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: Sep 09 12:09:14.881955 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618c796e-d820-439e-a9c5-6f3eb2e5c952 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:09:14.923359 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 754/1498] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:09:14 2022] POST /identity/v3/users => generated 288 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:09:14.927457 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d23b16-decf-47d0-92f5-166d2fd1e2b7 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:09:14.928943 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-28d23b16-decf-47d0-92f5-166d2fd1e2b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:09:14.990869 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d23b16-decf-47d0-92f5-166d2fd1e2b7 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:09:15.075110 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:09:15.086208 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7c8e86-bcb4-47e5-bd7d-9830f1e4cc95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:09:15.086807 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7c8e86-bcb4-47e5-bd7d-9830f1e4cc95 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:15.087111 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7c8e86-bcb4-47e5-bd7d-9830f1e4cc95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:15.087377 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7c8e86-bcb4-47e5-bd7d-9830f1e4cc95 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:15.088008 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7c8e86-bcb4-47e5-bd7d-9830f1e4cc95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:09:15.089007 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: WARNING py.warnings [None req-3d7c8e86-bcb4-47e5-bd7d-9830f1e4cc95 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 09 12:09:15.089007 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: warnings.warn(msg) Sep 09 12:09:15.089007 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: Sep 09 12:09:15.089895 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7c8e86-bcb4-47e5-bd7d-9830f1e4cc95 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:09:15.102602 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 745/1499] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:09:14 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:09:15.106763 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618c796e-d820-439e-a9c5-6f3eb2e5c952 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:09:15.109312 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-618c796e-d820-439e-a9c5-6f3eb2e5c952 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:09:15.121443 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618c796e-d820-439e-a9c5-6f3eb2e5c952 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:09:15.146062 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:09:15.217172 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78995ac8-281f-49d3-8bee-c30986649368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:09:15.218043 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78995ac8-281f-49d3-8bee-c30986649368 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:15.218336 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78995ac8-281f-49d3-8bee-c30986649368 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:15.218604 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78995ac8-281f-49d3-8bee-c30986649368 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:15.219282 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78995ac8-281f-49d3-8bee-c30986649368 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:09:15.220318 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: WARNING py.warnings [None req-78995ac8-281f-49d3-8bee-c30986649368 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 09 12:09:15.220318 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: warnings.warn(msg) Sep 09 12:09:15.220318 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: Sep 09 12:09:15.221215 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78995ac8-281f-49d3-8bee-c30986649368 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:09:15.234357 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 755/1500] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:09:15 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:09:15.238568 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7c8e86-bcb4-47e5-bd7d-9830f1e4cc95 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:09:15.240135 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-3d7c8e86-bcb4-47e5-bd7d-9830f1e4cc95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:09:15.273055 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7c8e86-bcb4-47e5-bd7d-9830f1e4cc95 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:09:15.366958 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc821074-50f3-430d-a315-cdcc26582482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:09:15.367625 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc821074-50f3-430d-a315-cdcc26582482 admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:15.367902 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc821074-50f3-430d-a315-cdcc26582482 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:15.368173 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc821074-50f3-430d-a315-cdcc26582482 admin admin] PATH_INFO: `/v3/projects/c5e8e844f1644022906dc9966227368f/users/1bcee81203fa483783abf1027c89eaee/roles/eecd2f160dd844f5906f3d345780c501` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:15.390441 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc821074-50f3-430d-a315-cdcc26582482 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5e8e844f1644022906dc9966227368f, user_id=1bcee81203fa483783abf1027c89eaee, role_id=eecd2f160dd844f5906f3d345780c501)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:09:15.391492 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: WARNING py.warnings [None req-bc821074-50f3-430d-a315-cdcc26582482 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 12:09:15.391492 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: warnings.warn(msg) Sep 09 12:09:15.391492 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: Sep 09 12:09:15.393456 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc821074-50f3-430d-a315-cdcc26582482 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:09:15.417405 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 746/1501] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:09:15 2022] PUT /identity/v3/projects/c5e8e844f1644022906dc9966227368f/users/1bcee81203fa483783abf1027c89eaee/roles/eecd2f160dd844f5906f3d345780c501 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:09:15.429730 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8114f1d2-60a0-4a64-a99a-06c19fe1c9e5 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:15.430017 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8114f1d2-60a0-4a64-a99a-06c19fe1c9e5 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:15.430294 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8114f1d2-60a0-4a64-a99a-06c19fe1c9e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:15.488515 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-8114f1d2-60a0-4a64-a99a-06c19fe1c9e5 None None] MFA Rules not processed for user `1bcee81203fa483783abf1027c89eaee`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:09:15.489666 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-8114f1d2-60a0-4a64-a99a-06c19fe1c9e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:09:15.566437 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 756/1502] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:09:15 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:09:15.580157 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acba9f36-d77a-457f-bf33-f949c277787a None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:15.580445 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acba9f36-d77a-457f-bf33-f949c277787a None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:15.580720 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acba9f36-d77a-457f-bf33-f949c277787a None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:15.582051 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 747/1503] 10.222.0.49 () {54 vars in 964 bytes} [Fri Sep 9 12:09:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:09:15.586673 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a479d242-2b96-43fc-ab30-4e5228927177 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:15.587007 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a479d242-2b96-43fc-ab30-4e5228927177 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:15.587269 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a479d242-2b96-43fc-ab30-4e5228927177 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:15.618533 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:09:15.664712 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-a479d242-2b96-43fc-ab30-4e5228927177 None None] MFA Rules not processed for user `535c258100df40b58a5ee44b2c21d621`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:09:15.665932 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-a479d242-2b96-43fc-ab30-4e5228927177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:09:15.740686 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 757/1504] 10.222.0.49 () {58 vars in 1061 bytes} [Fri Sep 9 12:09:15 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:09:15.748933 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acba9f36-d77a-457f-bf33-f949c277787a None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:09:15.750495 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-acba9f36-d77a-457f-bf33-f949c277787a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:09:15.800924 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:09:15.855316 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acba9f36-d77a-457f-bf33-f949c277787a None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:09:15.929258 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b42aa1-0821-4fa4-8061-4b0735c7e64c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535c258100df40b58a5ee44b2c21d621', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:09:15.929849 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b42aa1-0821-4fa4-8061-4b0735c7e64c service cinder] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:15.930117 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b42aa1-0821-4fa4-8061-4b0735c7e64c service cinder] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:15.930377 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b42aa1-0821-4fa4-8061-4b0735c7e64c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:15.931992 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-d1b42aa1-0821-4fa4-8061-4b0735c7e64c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:09:16.030111 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b42aa1-0821-4fa4-8061-4b0735c7e64c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:09:16.031650 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b42aa1-0821-4fa4-8061-4b0735c7e64c service cinder] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:09:16.035104 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:09:16.050851 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 748/1505] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:09:15 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:09:16.096881 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033e4db5-6021-48d0-a619-bdac2606cdbb None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:16.097169 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033e4db5-6021-48d0-a619-bdac2606cdbb None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:16.097444 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033e4db5-6021-48d0-a619-bdac2606cdbb None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:16.098694 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 758/1506] 10.222.0.49 () {52 vars in 830 bytes} [Fri Sep 9 12:09:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:09:16.102260 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b42aa1-0821-4fa4-8061-4b0735c7e64c service cinder] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:09:16.103533 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-d1b42aa1-0821-4fa4-8061-4b0735c7e64c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:09:16.203607 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b42aa1-0821-4fa4-8061-4b0735c7e64c service cinder] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:09:16.296098 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e747665b-a006-4c9a-b957-dd8c8572ab22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:09:16.296801 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e747665b-a006-4c9a-b957-dd8c8572ab22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:16.296945 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e747665b-a006-4c9a-b957-dd8c8572ab22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:16.297214 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e747665b-a006-4c9a-b957-dd8c8572ab22 admin admin] PATH_INFO: `/v3/users/1bcee81203fa483783abf1027c89eaee` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:16.309727 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e747665b-a006-4c9a-b957-dd8c8572ab22 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1bcee81203fa483783abf1027c89eaee)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:09:16.311080 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: WARNING py.warnings [None req-e747665b-a006-4c9a-b957-dd8c8572ab22 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 12:09:16.311080 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: warnings.warn(msg) Sep 09 12:09:16.311080 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: Sep 09 12:09:16.312297 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e747665b-a006-4c9a-b957-dd8c8572ab22 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:09:16.321783 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:09:16.379897 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-e747665b-a006-4c9a-b957-dd8c8572ab22 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1bcee81203fa483783abf1027c89eaee'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:16.403016 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-e747665b-a006-4c9a-b957-dd8c8572ab22 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1bcee81203fa483783abf1027c89eaee'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:16.417509 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-e747665b-a006-4c9a-b957-dd8c8572ab22 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1bcee81203fa483783abf1027c89eaee'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:16.428172 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-e747665b-a006-4c9a-b957-dd8c8572ab22 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1bcee81203fa483783abf1027c89eaee'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:16.429313 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-e747665b-a006-4c9a-b957-dd8c8572ab22 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1bcee81203fa483783abf1027c89eaee'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:16.435334 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 749/1507] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 9 12:09:16 2022] DELETE /identity/v3/users/1bcee81203fa483783abf1027c89eaee => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:09:16.440511 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88257bfe-af0d-4e72-b77a-44a077ccb702 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:16.440982 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88257bfe-af0d-4e72-b77a-44a077ccb702 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:16.441080 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88257bfe-af0d-4e72-b77a-44a077ccb702 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:16.507414 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-88257bfe-af0d-4e72-b77a-44a077ccb702 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:09:16.508749 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-88257bfe-af0d-4e72-b77a-44a077ccb702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:09:16.588103 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:09:16.600573 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 759/1508] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:09:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:09:16.610265 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e747665b-a006-4c9a-b957-dd8c8572ab22 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:09:16.611505 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-e747665b-a006-4c9a-b957-dd8c8572ab22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:09:16.718963 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e747665b-a006-4c9a-b957-dd8c8572ab22 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:09:16.772934 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:09:16.796317 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9b5358-ef92-409b-8d94-95c83c2440b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:09:16.796925 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9b5358-ef92-409b-8d94-95c83c2440b4 service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:16.797203 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9b5358-ef92-409b-8d94-95c83c2440b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:16.797470 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9b5358-ef92-409b-8d94-95c83c2440b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:16.799076 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-eb9b5358-ef92-409b-8d94-95c83c2440b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:09:16.906947 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9b5358-ef92-409b-8d94-95c83c2440b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:09:16.908285 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9b5358-ef92-409b-8d94-95c83c2440b4 service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:09:16.928166 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 750/1509] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:09:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:09:17.280641 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88257bfe-af0d-4e72-b77a-44a077ccb702 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:09:17.281864 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-88257bfe-af0d-4e72-b77a-44a077ccb702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:09:17.292974 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88257bfe-af0d-4e72-b77a-44a077ccb702 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:09:17.394188 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7444209c-1cc0-41e6-8e7d-75b5875ae6bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:09:17.394822 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7444209c-1cc0-41e6-8e7d-75b5875ae6bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:09:17.395122 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7444209c-1cc0-41e6-8e7d-75b5875ae6bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:09:17.395416 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7444209c-1cc0-41e6-8e7d-75b5875ae6bb admin admin] PATH_INFO: `/v3/projects/c5e8e844f1644022906dc9966227368f` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:09:17.405441 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7444209c-1cc0-41e6-8e7d-75b5875ae6bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5e8e844f1644022906dc9966227368f)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:09:17.406479 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: WARNING py.warnings [None req-7444209c-1cc0-41e6-8e7d-75b5875ae6bb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 12:09:17.406479 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: warnings.warn(msg) Sep 09 12:09:17.406479 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: Sep 09 12:09:17.407710 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7444209c-1cc0-41e6-8e7d-75b5875ae6bb admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:09:17.465280 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-7444209c-1cc0-41e6-8e7d-75b5875ae6bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5e8e844f1644022906dc9966227368f', 'request_id': 'req-7444209c-1cc0-41e6-8e7d-75b5875ae6bb'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:17.471765 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-7444209c-1cc0-41e6-8e7d-75b5875ae6bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5e8e844f1644022906dc9966227368f', 'request_id': 'req-7444209c-1cc0-41e6-8e7d-75b5875ae6bb'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:17.485046 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-7444209c-1cc0-41e6-8e7d-75b5875ae6bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5e8e844f1644022906dc9966227368f', 'request_id': 'req-7444209c-1cc0-41e6-8e7d-75b5875ae6bb'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:17.510162 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-7444209c-1cc0-41e6-8e7d-75b5875ae6bb admin admin] The token cache is being invalidate because project c5e8e844f1644022906dc9966227368f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81591) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:09:17.510451 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-7444209c-1cc0-41e6-8e7d-75b5875ae6bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:17.510996 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-7444209c-1cc0-41e6-8e7d-75b5875ae6bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:09:17.512294 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 760/1510] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 9 12:09:17 2022] DELETE /identity/v3/projects/c5e8e844f1644022906dc9966227368f => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:09:26.926057 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=62 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:09:27.508214 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:10:44.066494 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3454a7-03db-4bf7-83bc-17c5f6b8bc95 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:10:44.067114 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3454a7-03db-4bf7-83bc-17c5f6b8bc95 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:10:44.067114 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3454a7-03db-4bf7-83bc-17c5f6b8bc95 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:10:44.068431 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 751/1511] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 12:10:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:10:44.073384 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da95d9f-ad02-418e-ba22-6b05ff04edc2 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:10:44.073839 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da95d9f-ad02-418e-ba22-6b05ff04edc2 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:10:44.073946 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da95d9f-ad02-418e-ba22-6b05ff04edc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:10:44.156075 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-5da95d9f-ad02-418e-ba22-6b05ff04edc2 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:10:44.157276 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-5da95d9f-ad02-418e-ba22-6b05ff04edc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:10:44.258623 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 761/1512] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 12:10:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:10:44.414200 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3454a7-03db-4bf7-83bc-17c5f6b8bc95 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:10:44.416333 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-ab3454a7-03db-4bf7-83bc-17c5f6b8bc95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:10:44.535667 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3454a7-03db-4bf7-83bc-17c5f6b8bc95 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:10:44.542683 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963f2538-6c6a-4592-87a9-acc3cec3966f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70bac169d5cd43e1926f310344ac11fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:10:44.543349 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963f2538-6c6a-4592-87a9-acc3cec3966f service nova] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:10:44.543740 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963f2538-6c6a-4592-87a9-acc3cec3966f service nova] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:10:44.544014 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963f2538-6c6a-4592-87a9-acc3cec3966f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:10:44.552899 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963f2538-6c6a-4592-87a9-acc3cec3966f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:10:44.554481 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963f2538-6c6a-4592-87a9-acc3cec3966f service nova] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:10:44.564938 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 752/1513] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:10:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:10:45.663422 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7522cee9-4e70-452a-819e-61da4b47f1b8 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:10:45.663879 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7522cee9-4e70-452a-819e-61da4b47f1b8 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:10:45.663987 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7522cee9-4e70-452a-819e-61da4b47f1b8 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:10:45.665264 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 762/1514] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 12:10:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:10:45.671714 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c39b884-f1b2-4964-9715-65a2923a8899 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:10:45.672144 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c39b884-f1b2-4964-9715-65a2923a8899 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:10:45.672261 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c39b884-f1b2-4964-9715-65a2923a8899 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:10:45.692004 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-1c39b884-f1b2-4964-9715-65a2923a8899 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:10:45.693182 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-1c39b884-f1b2-4964-9715-65a2923a8899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:10:45.701573 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 753/1515] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 12:10:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:10:45.838217 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ea9ae8-671b-4284-bc67-ee69549cfb71 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:10:45.838500 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ea9ae8-671b-4284-bc67-ee69549cfb71 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:10:45.838770 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ea9ae8-671b-4284-bc67-ee69549cfb71 None None] PATH_INFO: `/v3` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:10:45.840077 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 763/1516] 10.222.0.71 () {54 vars in 925 bytes} [Fri Sep 9 12:10:45 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:10:45.846399 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d084ba05-cf87-4d38-8a68-9622da4258c0 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:10:45.846681 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d084ba05-cf87-4d38-8a68-9622da4258c0 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:10:45.846979 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d084ba05-cf87-4d38-8a68-9622da4258c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:10:45.857454 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ea9ae8-671b-4284-bc67-ee69549cfb71 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:10:45.872028 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ea9ae8-671b-4284-bc67-ee69549cfb71 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:10:45.877486 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f851f87-d8c5-49cf-b5a4-486592dc105f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70bac169d5cd43e1926f310344ac11fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:10:45.878081 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f851f87-d8c5-49cf-b5a4-486592dc105f service nova] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:10:45.878358 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f851f87-d8c5-49cf-b5a4-486592dc105f service nova] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:10:45.878631 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f851f87-d8c5-49cf-b5a4-486592dc105f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:10:45.886779 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f851f87-d8c5-49cf-b5a4-486592dc105f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:10:45.888998 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f851f87-d8c5-49cf-b5a4-486592dc105f service nova] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:10:45.898450 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 764/1517] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:10:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:10:45.907853 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-d084ba05-cf87-4d38-8a68-9622da4258c0 None None] MFA Rules not processed for user `d3adb2f0583c4ff782049dacdb3ee556`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:10:45.909206 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-d084ba05-cf87-4d38-8a68-9622da4258c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:10:45.967906 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:10:45.989106 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 754/1518] 10.222.0.71 () {58 vars in 1015 bytes} [Fri Sep 9 12:10:45 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:10:49.485835 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d424f2c5-bd3d-4cbe-b11b-76eb4146638c None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:10:49.486354 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d424f2c5-bd3d-4cbe-b11b-76eb4146638c None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:10:49.486465 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d424f2c5-bd3d-4cbe-b11b-76eb4146638c None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:10:49.487762 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 765/1519] 10.222.0.49 () {54 vars in 961 bytes} [Fri Sep 9 12:10:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:10:49.492845 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cb1290-b812-4c6f-bf8c-035075dc99df None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:10:49.493294 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cb1290-b812-4c6f-bf8c-035075dc99df None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:10:49.493400 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cb1290-b812-4c6f-bf8c-035075dc99df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:10:49.512862 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-b4cb1290-b812-4c6f-bf8c-035075dc99df None None] MFA Rules not processed for user `d3adb2f0583c4ff782049dacdb3ee556`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:10:49.514075 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-b4cb1290-b812-4c6f-bf8c-035075dc99df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:10:49.522357 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 755/1520] 10.222.0.49 () {58 vars in 1058 bytes} [Fri Sep 9 12:10:49 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:10:49.530707 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d424f2c5-bd3d-4cbe-b11b-76eb4146638c None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:10:49.544517 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d424f2c5-bd3d-4cbe-b11b-76eb4146638c None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:10:49.549405 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165eb618-03c3-4dee-8075-480335e1b8b1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3adb2f0583c4ff782049dacdb3ee556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:10:49.549993 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165eb618-03c3-4dee-8075-480335e1b8b1 service placement] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:10:49.550268 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165eb618-03c3-4dee-8075-480335e1b8b1 service placement] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:10:49.550535 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165eb618-03c3-4dee-8075-480335e1b8b1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:10:49.559202 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165eb618-03c3-4dee-8075-480335e1b8b1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:10:49.560761 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165eb618-03c3-4dee-8075-480335e1b8b1 service placement] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:10:49.570287 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 766/1521] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:10:49 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:10:57.350056 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a90375e-9fcc-4f33-9a3b-d309a59b93fb None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:10:57.350659 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a90375e-9fcc-4f33-9a3b-d309a59b93fb None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:10:57.350659 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a90375e-9fcc-4f33-9a3b-d309a59b93fb None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:10:57.352027 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 756/1522] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 12:10:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:10:57.358379 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba1a746-195c-4265-b4c4-fd30dfc1cd4a None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:10:57.358833 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba1a746-195c-4265-b4c4-fd30dfc1cd4a None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:10:57.358957 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba1a746-195c-4265-b4c4-fd30dfc1cd4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:10:57.380571 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-cba1a746-195c-4265-b4c4-fd30dfc1cd4a None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:10:57.381764 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-cba1a746-195c-4265-b4c4-fd30dfc1cd4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:10:57.390236 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 767/1523] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 12:10:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:10:57.545297 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a90375e-9fcc-4f33-9a3b-d309a59b93fb None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:10:57.559996 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a90375e-9fcc-4f33-9a3b-d309a59b93fb None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:10:57.565690 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bef65a-a512-4454-a5c0-71d657624c06 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70bac169d5cd43e1926f310344ac11fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:10:57.566285 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bef65a-a512-4454-a5c0-71d657624c06 service nova] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:10:57.566562 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bef65a-a512-4454-a5c0-71d657624c06 service nova] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:10:57.566830 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bef65a-a512-4454-a5c0-71d657624c06 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:10:57.575532 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bef65a-a512-4454-a5c0-71d657624c06 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:10:57.577104 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bef65a-a512-4454-a5c0-71d657624c06 service nova] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:10:57.586318 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 757/1524] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:10:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:10:58.659887 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5138fa-d331-4e1a-96d8-8a2d8fa972f7 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:10:58.660363 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5138fa-d331-4e1a-96d8-8a2d8fa972f7 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:10:58.660473 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5138fa-d331-4e1a-96d8-8a2d8fa972f7 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:10:58.661852 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 768/1525] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 12:10:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:10:58.668033 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a729221-8c45-42ca-b2b5-7f2e79b01dab None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:10:58.668495 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a729221-8c45-42ca-b2b5-7f2e79b01dab None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:10:58.668600 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a729221-8c45-42ca-b2b5-7f2e79b01dab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:10:58.688547 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-4a729221-8c45-42ca-b2b5-7f2e79b01dab None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:10:58.689754 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-4a729221-8c45-42ca-b2b5-7f2e79b01dab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:10:58.697518 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 758/1526] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 12:10:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:10:58.851381 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5138fa-d331-4e1a-96d8-8a2d8fa972f7 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:10:58.865998 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5138fa-d331-4e1a-96d8-8a2d8fa972f7 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:10:58.872275 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1ebe13-4c33-4e3d-aae5-b9f2003c749d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70bac169d5cd43e1926f310344ac11fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:10:58.872867 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1ebe13-4c33-4e3d-aae5-b9f2003c749d service nova] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:10:58.873145 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1ebe13-4c33-4e3d-aae5-b9f2003c749d service nova] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:10:58.873419 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1ebe13-4c33-4e3d-aae5-b9f2003c749d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:10:58.881317 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1ebe13-4c33-4e3d-aae5-b9f2003c749d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:10:58.882876 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1ebe13-4c33-4e3d-aae5-b9f2003c749d service nova] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:10:58.892293 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 769/1527] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:10:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:11:02.869615 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce723e83-067a-451c-b094-a9964e24ae90 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:11:02.870131 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce723e83-067a-451c-b094-a9964e24ae90 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:11:02.870131 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce723e83-067a-451c-b094-a9964e24ae90 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:11:02.871521 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 759/1528] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 12:11:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:11:02.876628 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a68f12-28ef-4b57-a408-e056f2dd9015 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:11:02.877071 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a68f12-28ef-4b57-a408-e056f2dd9015 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:11:02.877178 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a68f12-28ef-4b57-a408-e056f2dd9015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:11:02.896755 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-06a68f12-28ef-4b57-a408-e056f2dd9015 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:11:02.897930 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-06a68f12-28ef-4b57-a408-e056f2dd9015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:11:02.906464 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 770/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 12:11:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:11:03.194066 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce723e83-067a-451c-b094-a9964e24ae90 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:11:03.195813 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-ce723e83-067a-451c-b094-a9964e24ae90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:11:03.273147 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce723e83-067a-451c-b094-a9964e24ae90 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:11:03.276791 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17781085-25c9-43d0-ab4f-dde09bae9796 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:11:03.277490 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17781085-25c9-43d0-ab4f-dde09bae9796 service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:11:03.277854 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17781085-25c9-43d0-ab4f-dde09bae9796 service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:11:03.278126 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17781085-25c9-43d0-ab4f-dde09bae9796 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:11:03.286956 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17781085-25c9-43d0-ab4f-dde09bae9796 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:11:03.288406 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17781085-25c9-43d0-ab4f-dde09bae9796 service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:11:03.297747 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 760/1530] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:11:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:11:05.468024 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acd8742-4820-4026-99b5-b66563693a41 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:11:05.468591 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acd8742-4820-4026-99b5-b66563693a41 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:11:05.468591 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acd8742-4820-4026-99b5-b66563693a41 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:11:05.469950 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 771/1531] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 12:11:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:11:05.475180 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641e6099-3664-42b9-81ad-b4eeabf681c3 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:11:05.475603 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641e6099-3664-42b9-81ad-b4eeabf681c3 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:11:05.475700 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641e6099-3664-42b9-81ad-b4eeabf681c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:11:05.496229 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-641e6099-3664-42b9-81ad-b4eeabf681c3 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:11:05.497777 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-641e6099-3664-42b9-81ad-b4eeabf681c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:11:05.505419 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 761/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 12:11:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:11:05.794840 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acd8742-4820-4026-99b5-b66563693a41 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:11:05.805308 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acd8742-4820-4026-99b5-b66563693a41 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:11:05.809731 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406d565b-b114-4457-8a1b-a9eb7fc11a2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:11:05.810322 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406d565b-b114-4457-8a1b-a9eb7fc11a2a service neutron] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:11:05.810596 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406d565b-b114-4457-8a1b-a9eb7fc11a2a service neutron] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:11:05.810862 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406d565b-b114-4457-8a1b-a9eb7fc11a2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:11:05.819732 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406d565b-b114-4457-8a1b-a9eb7fc11a2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:11:05.821170 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406d565b-b114-4457-8a1b-a9eb7fc11a2a service neutron] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:11:05.830323 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 772/1533] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:11:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:11:09.587296 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf973edb-cf01-49f1-9a5f-414be68cc796 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:11:09.587886 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf973edb-cf01-49f1-9a5f-414be68cc796 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:11:09.587886 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf973edb-cf01-49f1-9a5f-414be68cc796 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:11:09.589227 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 762/1534] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 12:11:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:11:09.601178 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7bb5a6-b42a-4b58-953b-005a40222fa9 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:11:09.601632 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7bb5a6-b42a-4b58-953b-005a40222fa9 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:11:09.601743 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7bb5a6-b42a-4b58-953b-005a40222fa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:11:09.609927 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:11:09.623235 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-4d7bb5a6-b42a-4b58-953b-005a40222fa9 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:11:09.624427 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-4d7bb5a6-b42a-4b58-953b-005a40222fa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:11:09.632177 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 773/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 12:11:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:11:09.934434 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf973edb-cf01-49f1-9a5f-414be68cc796 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:11:09.936111 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-bf973edb-cf01-49f1-9a5f-414be68cc796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:11:10.013320 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf973edb-cf01-49f1-9a5f-414be68cc796 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:11:10.016827 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002e0acb-9402-4f92-8e7b-853d6c886bc8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535c258100df40b58a5ee44b2c21d621', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:11:10.017420 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002e0acb-9402-4f92-8e7b-853d6c886bc8 service cinder] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:11:10.017694 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002e0acb-9402-4f92-8e7b-853d6c886bc8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:11:10.017960 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002e0acb-9402-4f92-8e7b-853d6c886bc8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:11:10.026549 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002e0acb-9402-4f92-8e7b-853d6c886bc8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:11:10.028374 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002e0acb-9402-4f92-8e7b-853d6c886bc8 service cinder] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:11:10.037578 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 763/1536] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:11:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:11:12.193711 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e26afab-a80f-451b-a5fb-97674e9601e2 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:11:12.194247 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e26afab-a80f-451b-a5fb-97674e9601e2 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:11:12.194413 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e26afab-a80f-451b-a5fb-97674e9601e2 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:11:12.195869 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 774/1537] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 12:11:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:11:12.201250 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0e70ae-5741-495f-a6b5-d8a2a02e9d0c None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:11:12.201684 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0e70ae-5741-495f-a6b5-d8a2a02e9d0c None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:11:12.201790 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0e70ae-5741-495f-a6b5-d8a2a02e9d0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:11:12.222199 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-9c0e70ae-5741-495f-a6b5-d8a2a02e9d0c None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:11:12.223433 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-9c0e70ae-5741-495f-a6b5-d8a2a02e9d0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:11:12.231640 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 764/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 12:11:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:11:12.543805 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e26afab-a80f-451b-a5fb-97674e9601e2 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:11:12.545667 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-8e26afab-a80f-451b-a5fb-97674e9601e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:11:12.558390 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e26afab-a80f-451b-a5fb-97674e9601e2 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:11:12.562277 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f56154-16b9-4b29-8fd1-6f8851ecdec1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535c258100df40b58a5ee44b2c21d621', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:11:12.562865 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f56154-16b9-4b29-8fd1-6f8851ecdec1 service cinder] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:11:12.563165 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f56154-16b9-4b29-8fd1-6f8851ecdec1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:11:12.563430 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f56154-16b9-4b29-8fd1-6f8851ecdec1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:11:12.572288 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f56154-16b9-4b29-8fd1-6f8851ecdec1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:11:12.574804 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f56154-16b9-4b29-8fd1-6f8851ecdec1 service cinder] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:11:12.584145 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 775/1539] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:11:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:11:22.217318 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=92 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:11:22.580115 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=8 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:12:37.523653 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88413b4b-1423-46a3-8e68-e1c52ad2eee0 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:37.524354 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88413b4b-1423-46a3-8e68-e1c52ad2eee0 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:37.524354 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88413b4b-1423-46a3-8e68-e1c52ad2eee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:37.553900 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-88413b4b-1423-46a3-8e68-e1c52ad2eee0 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:12:37.555188 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-88413b4b-1423-46a3-8e68-e1c52ad2eee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:12:37.567175 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 765/1540] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:12:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:12:37.577215 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c56b7d2-e6eb-4903-b83d-14519fdc09e4 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:37.577829 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c56b7d2-e6eb-4903-b83d-14519fdc09e4 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:37.577829 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c56b7d2-e6eb-4903-b83d-14519fdc09e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:37.611451 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-3c56b7d2-e6eb-4903-b83d-14519fdc09e4 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:12:37.612722 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-3c56b7d2-e6eb-4903-b83d-14519fdc09e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:12:37.626099 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 776/1541] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:12:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:12:37.631195 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88413b4b-1423-46a3-8e68-e1c52ad2eee0 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:12:37.649171 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88413b4b-1423-46a3-8e68-e1c52ad2eee0 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:12:37.663678 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0aba3c-4474-4fd0-bdcb-a9ebd2452eb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:12:37.664321 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0aba3c-4474-4fd0-bdcb-a9ebd2452eb9 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:37.664605 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0aba3c-4474-4fd0-bdcb-a9ebd2452eb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:37.664880 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0aba3c-4474-4fd0-bdcb-a9ebd2452eb9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:37.665544 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0aba3c-4474-4fd0-bdcb-a9ebd2452eb9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:12:37.665846 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0aba3c-4474-4fd0-bdcb-a9ebd2452eb9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:12:37.667299 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: WARNING py.warnings [None req-8b0aba3c-4474-4fd0-bdcb-a9ebd2452eb9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 09 12:12:37.667299 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: warnings.warn(msg) Sep 09 12:12:37.667299 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: Sep 09 12:12:37.668295 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0aba3c-4474-4fd0-bdcb-a9ebd2452eb9 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:12:37.688163 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 766/1542] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 9 12:12:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:12:37.696971 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c56b7d2-e6eb-4903-b83d-14519fdc09e4 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:12:37.709201 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c56b7d2-e6eb-4903-b83d-14519fdc09e4 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:12:37.717395 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd22447-cdcf-4d22-a143-2d5073bc56df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:12:37.718012 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd22447-cdcf-4d22-a143-2d5073bc56df admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:37.718288 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd22447-cdcf-4d22-a143-2d5073bc56df admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:37.718553 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd22447-cdcf-4d22-a143-2d5073bc56df admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:37.719213 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd22447-cdcf-4d22-a143-2d5073bc56df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:12:37.720224 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: WARNING py.warnings [None req-9bd22447-cdcf-4d22-a143-2d5073bc56df admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 12:12:37.720224 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: warnings.warn(msg) Sep 09 12:12:37.720224 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: Sep 09 12:12:37.721411 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd22447-cdcf-4d22-a143-2d5073bc56df admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:12:37.726586 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29c6425-f6ca-4111-a6e4-25873c19dd51 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:37.726873 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29c6425-f6ca-4111-a6e4-25873c19dd51 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:37.727176 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29c6425-f6ca-4111-a6e4-25873c19dd51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:37.763028 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-b29c6425-f6ca-4111-a6e4-25873c19dd51 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:12:37.763028 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-b29c6425-f6ca-4111-a6e4-25873c19dd51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:12:37.773309 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 767/1543] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:12:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:12:37.774810 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5324fb-d488-4858-a12b-45bb58727e70 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:37.778929 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5324fb-d488-4858-a12b-45bb58727e70 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:37.779235 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5324fb-d488-4858-a12b-45bb58727e70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:37.787077 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 777/1544] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 9 12:12:37 2022] POST /identity/v3/projects => generated 383 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:12:37.788523 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec0456c-0369-400f-bb2f-13c86bde1cb2 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:37.788810 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec0456c-0369-400f-bb2f-13c86bde1cb2 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:37.789079 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec0456c-0369-400f-bb2f-13c86bde1cb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:37.814562 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-4f5324fb-d488-4858-a12b-45bb58727e70 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:12:37.815977 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-4f5324fb-d488-4858-a12b-45bb58727e70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:12:37.818392 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-3ec0456c-0369-400f-bb2f-13c86bde1cb2 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:12:37.824642 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-3ec0456c-0369-400f-bb2f-13c86bde1cb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:12:37.906638 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 778/1545] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:12:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:12:37.907523 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec0456c-0369-400f-bb2f-13c86bde1cb2 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:12:37.923778 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec0456c-0369-400f-bb2f-13c86bde1cb2 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:12:37.936904 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd516095-ff5b-41c9-b82a-ba25919e5cd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:12:37.939389 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd516095-ff5b-41c9-b82a-ba25919e5cd6 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:37.939891 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd516095-ff5b-41c9-b82a-ba25919e5cd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:37.940407 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd516095-ff5b-41c9-b82a-ba25919e5cd6 admin admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:37.940855 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 768/1546] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:12:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:12:37.942456 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54269be-d5d9-4992-93e1-effb4e06cc62 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:37.943385 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd516095-ff5b-41c9-b82a-ba25919e5cd6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:12:37.950947 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd516095-ff5b-41c9-b82a-ba25919e5cd6 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:12:37.952007 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54269be-d5d9-4992-93e1-effb4e06cc62 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:37.952531 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54269be-d5d9-4992-93e1-effb4e06cc62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:37.979960 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-e54269be-d5d9-4992-93e1-effb4e06cc62 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:12:37.981167 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 779/1547] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:12:37 2022] POST /identity/v3/users => generated 292 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:12:37.981704 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-e54269be-d5d9-4992-93e1-effb4e06cc62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:12:37.982309 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd516095-ff5b-41c9-b82a-ba25919e5cd6 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:12:37.989828 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 769/1548] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:12:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:12:37.991056 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a9f503-cb0c-4387-9fe0-c8350e22d292 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:37.991340 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a9f503-cb0c-4387-9fe0-c8350e22d292 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:37.991610 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a9f503-cb0c-4387-9fe0-c8350e22d292 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:37.992689 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd516095-ff5b-41c9-b82a-ba25919e5cd6 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:12:37.999250 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775751c6-4cab-4826-b092-84d0bd5a59d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:12:37.999856 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775751c6-4cab-4826-b092-84d0bd5a59d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:38.000137 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775751c6-4cab-4826-b092-84d0bd5a59d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:38.000435 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775751c6-4cab-4826-b092-84d0bd5a59d9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:38.001082 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775751c6-4cab-4826-b092-84d0bd5a59d9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:12:38.001383 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775751c6-4cab-4826-b092-84d0bd5a59d9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:12:38.003028 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775751c6-4cab-4826-b092-84d0bd5a59d9 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:12:38.011683 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-d1a9f503-cb0c-4387-9fe0-c8350e22d292 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:12:38.012883 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-d1a9f503-cb0c-4387-9fe0-c8350e22d292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:12:38.015412 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 780/1549] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 9 12:12:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:12:38.016736 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419a0ce6-9797-439a-9f38-c59a3076b3a8 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:38.017023 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419a0ce6-9797-439a-9f38-c59a3076b3a8 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:38.017294 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419a0ce6-9797-439a-9f38-c59a3076b3a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:38.026847 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 770/1550] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:12:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:12:38.028270 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fd788b-50c3-45b2-be9e-0aa9d2f63f7e None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:38.028603 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fd788b-50c3-45b2-be9e-0aa9d2f63f7e None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:38.028906 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fd788b-50c3-45b2-be9e-0aa9d2f63f7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:38.036892 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-419a0ce6-9797-439a-9f38-c59a3076b3a8 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:12:38.038096 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-419a0ce6-9797-439a-9f38-c59a3076b3a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:12:38.046756 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 781/1551] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:12:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:12:38.047983 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e042ceee-93c4-452d-8758-2922dcc0983e None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:38.048259 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e042ceee-93c4-452d-8758-2922dcc0983e None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:38.048525 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e042ceee-93c4-452d-8758-2922dcc0983e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:38.048792 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-82fd788b-50c3-45b2-be9e-0aa9d2f63f7e None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:12:38.050251 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-82fd788b-50c3-45b2-be9e-0aa9d2f63f7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:12:38.059161 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 771/1552] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:12:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:12:38.060500 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775a0116-c0b9-44d7-b0a8-4ce8a9a12fd9 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:38.060788 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775a0116-c0b9-44d7-b0a8-4ce8a9a12fd9 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:38.061321 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775a0116-c0b9-44d7-b0a8-4ce8a9a12fd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:38.069280 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-e042ceee-93c4-452d-8758-2922dcc0983e None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:12:38.070592 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-e042ceee-93c4-452d-8758-2922dcc0983e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:12:38.079439 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 782/1553] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:12:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:12:38.080216 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e042ceee-93c4-452d-8758-2922dcc0983e None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:12:38.087608 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-775a0116-c0b9-44d7-b0a8-4ce8a9a12fd9 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:12:38.088807 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-775a0116-c0b9-44d7-b0a8-4ce8a9a12fd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:12:38.089476 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e042ceee-93c4-452d-8758-2922dcc0983e None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:12:38.094987 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c540bf-b107-4839-a4be-8ab3dbb97f77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:12:38.095572 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c540bf-b107-4839-a4be-8ab3dbb97f77 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:38.095847 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c540bf-b107-4839-a4be-8ab3dbb97f77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:38.096118 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c540bf-b107-4839-a4be-8ab3dbb97f77 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:38.096746 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c540bf-b107-4839-a4be-8ab3dbb97f77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:12:38.097425 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 772/1554] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:12:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:12:38.098326 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c540bf-b107-4839-a4be-8ab3dbb97f77 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:12:38.104229 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8b135b-c250-4efd-b3f3-fe2ba9c374f0 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:38.104608 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8b135b-c250-4efd-b3f3-fe2ba9c374f0 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:38.105105 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8b135b-c250-4efd-b3f3-fe2ba9c374f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:38.109254 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 783/1555] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:12:38 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:12:38.110026 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c540bf-b107-4839-a4be-8ab3dbb97f77 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:12:38.119801 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c540bf-b107-4839-a4be-8ab3dbb97f77 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:12:38.124757 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-8c8b135b-c250-4efd-b3f3-fe2ba9c374f0 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:12:38.125455 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b25205-e8cc-44a4-9ab3-63a5f371daa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:12:38.126028 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-8c8b135b-c250-4efd-b3f3-fe2ba9c374f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:12:38.126151 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b25205-e8cc-44a4-9ab3-63a5f371daa6 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:38.126300 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b25205-e8cc-44a4-9ab3-63a5f371daa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:38.126566 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b25205-e8cc-44a4-9ab3-63a5f371daa6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:38.127213 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b25205-e8cc-44a4-9ab3-63a5f371daa6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:12:38.129076 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b25205-e8cc-44a4-9ab3-63a5f371daa6 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:12:38.139139 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 773/1556] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:12:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:12:38.139139 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dca5701-34ce-4e14-8bda-afd93179ceb5 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:38.139139 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dca5701-34ce-4e14-8bda-afd93179ceb5 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:38.139139 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dca5701-34ce-4e14-8bda-afd93179ceb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:38.158696 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-6dca5701-34ce-4e14-8bda-afd93179ceb5 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:12:38.159973 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-6dca5701-34ce-4e14-8bda-afd93179ceb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:12:38.160927 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 784/1557] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 9 12:12:38 2022] POST /identity/v3/projects => generated 379 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:12:38.162252 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7f3ed9-22e4-4d8e-b6c6-cabc38266653 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:38.162540 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7f3ed9-22e4-4d8e-b6c6-cabc38266653 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:38.162813 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7f3ed9-22e4-4d8e-b6c6-cabc38266653 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:38.182493 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-bd7f3ed9-22e4-4d8e-b6c6-cabc38266653 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:12:38.184474 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-bd7f3ed9-22e4-4d8e-b6c6-cabc38266653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:12:38.220554 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 774/1558] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:12:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:12:38.221474 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dca5701-34ce-4e14-8bda-afd93179ceb5 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:12:38.227737 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:12:38.231541 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dca5701-34ce-4e14-8bda-afd93179ceb5 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:12:38.236457 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a63b68-8498-4e17-a596-2a6ff5f1b616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:12:38.237303 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a63b68-8498-4e17-a596-2a6ff5f1b616 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:38.237587 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a63b68-8498-4e17-a596-2a6ff5f1b616 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:38.238175 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a63b68-8498-4e17-a596-2a6ff5f1b616 admin admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:38.238828 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a63b68-8498-4e17-a596-2a6ff5f1b616 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:12:38.239168 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a63b68-8498-4e17-a596-2a6ff5f1b616 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:12:38.240815 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a63b68-8498-4e17-a596-2a6ff5f1b616 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:12:38.248796 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 785/1559] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:12:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:12:38.250333 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed37951-2fd8-4372-9ad6-b0feedccd295 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:38.250774 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed37951-2fd8-4372-9ad6-b0feedccd295 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:38.251068 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed37951-2fd8-4372-9ad6-b0feedccd295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:38.255185 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 775/1560] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 9 12:12:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:12:38.256417 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b10b112-83c5-48d8-aa00-96269896ec81 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:38.256709 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b10b112-83c5-48d8-aa00-96269896ec81 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:38.256983 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b10b112-83c5-48d8-aa00-96269896ec81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:38.274336 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-1ed37951-2fd8-4372-9ad6-b0feedccd295 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:12:38.275555 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-1ed37951-2fd8-4372-9ad6-b0feedccd295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:12:38.278047 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-2b10b112-83c5-48d8-aa00-96269896ec81 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:12:38.279207 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-2b10b112-83c5-48d8-aa00-96269896ec81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:12:38.283901 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 786/1561] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:12:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:12:38.286769 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed37951-2fd8-4372-9ad6-b0feedccd295 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:12:38.287452 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 776/1562] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:12:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:12:38.288233 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b10b112-83c5-48d8-aa00-96269896ec81 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:12:38.296922 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed37951-2fd8-4372-9ad6-b0feedccd295 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:12:38.297620 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b10b112-83c5-48d8-aa00-96269896ec81 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:12:38.301783 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b31f50-739e-47a1-8439-ce27316b45b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:12:38.302184 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12746af0-adcd-4b65-9f7e-94da198dba89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:12:38.302384 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b31f50-739e-47a1-8439-ce27316b45b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:38.302665 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b31f50-739e-47a1-8439-ce27316b45b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:38.302779 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12746af0-adcd-4b65-9f7e-94da198dba89 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:38.302966 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b31f50-739e-47a1-8439-ce27316b45b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:38.303129 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12746af0-adcd-4b65-9f7e-94da198dba89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:38.303328 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12746af0-adcd-4b65-9f7e-94da198dba89 admin admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:38.303608 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b31f50-739e-47a1-8439-ce27316b45b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:12:38.303961 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12746af0-adcd-4b65-9f7e-94da198dba89 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:12:38.304284 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12746af0-adcd-4b65-9f7e-94da198dba89 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:12:38.305208 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b31f50-739e-47a1-8439-ce27316b45b1 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:12:38.305868 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12746af0-adcd-4b65-9f7e-94da198dba89 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:12:38.315975 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 787/1563] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:12:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:12:38.316754 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b31f50-739e-47a1-8439-ce27316b45b1 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:12:38.319225 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 777/1564] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 9 12:12:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:12:38.320086 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12746af0-adcd-4b65-9f7e-94da198dba89 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:12:38.327800 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b31f50-739e-47a1-8439-ce27316b45b1 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:12:38.330768 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12746af0-adcd-4b65-9f7e-94da198dba89 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:12:38.332508 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6cb8df-26ad-4a3b-8be0-3ab469fd34af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:12:38.333109 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6cb8df-26ad-4a3b-8be0-3ab469fd34af admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:38.333420 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6cb8df-26ad-4a3b-8be0-3ab469fd34af admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:38.333687 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6cb8df-26ad-4a3b-8be0-3ab469fd34af admin admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:38.334309 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6cb8df-26ad-4a3b-8be0-3ab469fd34af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:12:38.335317 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291d1397-d972-43b7-ae44-821669459aa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:12:38.335900 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291d1397-d972-43b7-ae44-821669459aa7 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:38.336175 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291d1397-d972-43b7-ae44-821669459aa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:38.336283 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6cb8df-26ad-4a3b-8be0-3ab469fd34af admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:12:38.336443 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291d1397-d972-43b7-ae44-821669459aa7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:38.337746 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291d1397-d972-43b7-ae44-821669459aa7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:12:38.338048 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291d1397-d972-43b7-ae44-821669459aa7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:12:38.340298 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291d1397-d972-43b7-ae44-821669459aa7 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:12:38.342733 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:12:38.354201 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 778/1565] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 9 12:12:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:12:38.354925 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291d1397-d972-43b7-ae44-821669459aa7 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:12:38.363710 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 788/1566] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:12:38 2022] POST /identity/v3/users => generated 290 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:12:38.363710 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6cb8df-26ad-4a3b-8be0-3ab469fd34af admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:12:38.366155 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291d1397-d972-43b7-ae44-821669459aa7 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:12:38.371251 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e7dba5-f9b4-4324-85cf-5bbf6bedef08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:12:38.371698 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6cb8df-26ad-4a3b-8be0-3ab469fd34af admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:12:38.371973 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e7dba5-f9b4-4324-85cf-5bbf6bedef08 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:38.372260 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e7dba5-f9b4-4324-85cf-5bbf6bedef08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:38.372634 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e7dba5-f9b4-4324-85cf-5bbf6bedef08 admin admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:38.373285 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e7dba5-f9b4-4324-85cf-5bbf6bedef08 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:12:38.373589 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e7dba5-f9b4-4324-85cf-5bbf6bedef08 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:12:38.375366 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e7dba5-f9b4-4324-85cf-5bbf6bedef08 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:12:38.376330 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43036af-9630-45f5-870b-597392ef13f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:12:38.376916 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43036af-9630-45f5-870b-597392ef13f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:38.377195 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43036af-9630-45f5-870b-597392ef13f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:38.377463 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43036af-9630-45f5-870b-597392ef13f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:38.378082 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43036af-9630-45f5-870b-597392ef13f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:12:38.380001 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43036af-9630-45f5-870b-597392ef13f6 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:12:38.387136 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 779/1567] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 9 12:12:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:12:38.387935 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e7dba5-f9b4-4324-85cf-5bbf6bedef08 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:12:38.397499 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e7dba5-f9b4-4324-85cf-5bbf6bedef08 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:12:38.403447 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4079186b-64f6-4cac-9f03-2485a89c056a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:12:38.404038 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4079186b-64f6-4cac-9f03-2485a89c056a admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:38.404312 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4079186b-64f6-4cac-9f03-2485a89c056a admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:38.404662 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4079186b-64f6-4cac-9f03-2485a89c056a admin admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:38.405452 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4079186b-64f6-4cac-9f03-2485a89c056a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:12:38.405750 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4079186b-64f6-4cac-9f03-2485a89c056a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:12:38.407370 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4079186b-64f6-4cac-9f03-2485a89c056a admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:12:38.409531 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 789/1568] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 9 12:12:38 2022] POST /identity/v3/projects => generated 371 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:12:38.410430 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43036af-9630-45f5-870b-597392ef13f6 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:12:38.423864 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 780/1569] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 9 12:12:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:12:38.424554 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4079186b-64f6-4cac-9f03-2485a89c056a admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:12:38.433843 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4079186b-64f6-4cac-9f03-2485a89c056a admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:12:38.482100 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43036af-9630-45f5-870b-597392ef13f6 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:12:38.487129 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188560e8-9d7f-4402-9a66-b858154ecfc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:12:38.487743 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188560e8-9d7f-4402-9a66-b858154ecfc9 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:38.488022 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188560e8-9d7f-4402-9a66-b858154ecfc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:38.488293 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188560e8-9d7f-4402-9a66-b858154ecfc9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:38.488939 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188560e8-9d7f-4402-9a66-b858154ecfc9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:12:38.489240 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188560e8-9d7f-4402-9a66-b858154ecfc9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:12:38.490864 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188560e8-9d7f-4402-9a66-b858154ecfc9 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:12:38.494069 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d148aa-08b0-4df5-893f-489e4cee6f88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:12:38.494791 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d148aa-08b0-4df5-893f-489e4cee6f88 admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:38.495210 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d148aa-08b0-4df5-893f-489e4cee6f88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:38.495506 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d148aa-08b0-4df5-893f-489e4cee6f88 admin admin] PATH_INFO: `/v3/projects/30fa27304a4f4d4ab05c7763bc346d57/users/2e125f66d9c84e9797099065351a07da/roles/eecd2f160dd844f5906f3d345780c501` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:38.505187 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 790/1570] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 9 12:12:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:12:38.506035 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188560e8-9d7f-4402-9a66-b858154ecfc9 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:12:38.516586 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d148aa-08b0-4df5-893f-489e4cee6f88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=30fa27304a4f4d4ab05c7763bc346d57, user_id=2e125f66d9c84e9797099065351a07da, role_id=eecd2f160dd844f5906f3d345780c501)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:12:38.516781 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188560e8-9d7f-4402-9a66-b858154ecfc9 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:12:38.519310 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d148aa-08b0-4df5-893f-489e4cee6f88 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:12:38.521216 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b70870b-8ff5-48b3-a936-2b9ec81259c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:12:38.521977 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b70870b-8ff5-48b3-a936-2b9ec81259c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:38.522639 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b70870b-8ff5-48b3-a936-2b9ec81259c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:38.522954 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b70870b-8ff5-48b3-a936-2b9ec81259c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:38.523660 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b70870b-8ff5-48b3-a936-2b9ec81259c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:12:38.525955 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b70870b-8ff5-48b3-a936-2b9ec81259c0 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:12:38.536665 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 781/1571] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:12:38 2022] PUT /identity/v3/projects/30fa27304a4f4d4ab05c7763bc346d57/users/2e125f66d9c84e9797099065351a07da/roles/eecd2f160dd844f5906f3d345780c501 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:12:38.536665 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d148aa-08b0-4df5-893f-489e4cee6f88 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:12:38.555054 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 791/1572] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 9 12:12:38 2022] POST /identity/v3/projects => generated 367 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:12:38.555874 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b70870b-8ff5-48b3-a936-2b9ec81259c0 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:12:38.599222 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d148aa-08b0-4df5-893f-489e4cee6f88 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:12:38.619317 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b70870b-8ff5-48b3-a936-2b9ec81259c0 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:12:38.624088 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ab32d2-924e-4557-ba5c-409411dffb42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:12:38.624701 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ab32d2-924e-4557-ba5c-409411dffb42 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:38.624981 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ab32d2-924e-4557-ba5c-409411dffb42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:38.625249 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ab32d2-924e-4557-ba5c-409411dffb42 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:38.625885 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ab32d2-924e-4557-ba5c-409411dffb42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:12:38.627569 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ab32d2-924e-4557-ba5c-409411dffb42 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:12:38.629672 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:12:38.642943 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 792/1573] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:12:38 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:12:38.642943 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ab32d2-924e-4557-ba5c-409411dffb42 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:12:38.652021 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ab32d2-924e-4557-ba5c-409411dffb42 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:12:38.657490 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9edb8b-493e-491c-b973-1cde5f8ef89c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:12:38.658083 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9edb8b-493e-491c-b973-1cde5f8ef89c admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:38.658366 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9edb8b-493e-491c-b973-1cde5f8ef89c admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:38.658630 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9edb8b-493e-491c-b973-1cde5f8ef89c admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:38.658749 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2334a2c3-3988-4eee-aa3b-bfb774a66749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:12:38.659273 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2334a2c3-3988-4eee-aa3b-bfb774a66749 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:38.659427 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9edb8b-493e-491c-b973-1cde5f8ef89c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:12:38.659555 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2334a2c3-3988-4eee-aa3b-bfb774a66749 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:38.659827 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2334a2c3-3988-4eee-aa3b-bfb774a66749 admin admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:38.660451 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2334a2c3-3988-4eee-aa3b-bfb774a66749 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:12:38.661186 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9edb8b-493e-491c-b973-1cde5f8ef89c admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:12:38.662344 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2334a2c3-3988-4eee-aa3b-bfb774a66749 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:12:38.666850 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:12:38.689599 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 793/1574] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 9 12:12:38 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:12:38.691656 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 782/1575] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 9 12:12:38 2022] POST /identity/v3/projects => generated 393 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:12:38.693693 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2334a2c3-3988-4eee-aa3b-bfb774a66749 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:12:38.693828 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9edb8b-493e-491c-b973-1cde5f8ef89c admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:12:38.755118 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2334a2c3-3988-4eee-aa3b-bfb774a66749 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:12:38.756145 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9edb8b-493e-491c-b973-1cde5f8ef89c admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:12:38.760023 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5c5380-d3e3-4122-ad3f-250b1043dcb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:12:38.760704 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5c5380-d3e3-4122-ad3f-250b1043dcb3 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:38.760887 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1947e1a9-2b79-4da2-8e78-2c362e0ac8c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:12:38.761036 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5c5380-d3e3-4122-ad3f-250b1043dcb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:38.761272 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5c5380-d3e3-4122-ad3f-250b1043dcb3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:38.762085 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1947e1a9-2b79-4da2-8e78-2c362e0ac8c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:38.762319 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1947e1a9-2b79-4da2-8e78-2c362e0ac8c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:38.762653 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5c5380-d3e3-4122-ad3f-250b1043dcb3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:12:38.762769 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1947e1a9-2b79-4da2-8e78-2c362e0ac8c1 admin admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:38.763358 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1947e1a9-2b79-4da2-8e78-2c362e0ac8c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:12:38.764605 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5c5380-d3e3-4122-ad3f-250b1043dcb3 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:12:38.765318 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1947e1a9-2b79-4da2-8e78-2c362e0ac8c1 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:12:38.786803 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 794/1576] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:12:38 2022] POST /identity/v3/users => generated 286 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:12:38.787669 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1947e1a9-2b79-4da2-8e78-2c362e0ac8c1 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:12:38.792738 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 783/1577] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 9 12:12:38 2022] POST /identity/v3/projects => generated 367 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:12:38.794407 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680b4709-0b9f-4e11-b3e5-5f49f84a9406 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:38.794690 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680b4709-0b9f-4e11-b3e5-5f49f84a9406 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:38.794990 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680b4709-0b9f-4e11-b3e5-5f49f84a9406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:38.797750 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1947e1a9-2b79-4da2-8e78-2c362e0ac8c1 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:12:38.814684 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-680b4709-0b9f-4e11-b3e5-5f49f84a9406 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:12:38.815921 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-680b4709-0b9f-4e11-b3e5-5f49f84a9406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:12:38.861192 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28317c1-d625-4237-ba4e-8b1545d7df3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:12:38.861802 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28317c1-d625-4237-ba4e-8b1545d7df3d admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:38.862077 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28317c1-d625-4237-ba4e-8b1545d7df3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:38.862344 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28317c1-d625-4237-ba4e-8b1545d7df3d admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:38.863075 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28317c1-d625-4237-ba4e-8b1545d7df3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:12:38.864980 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28317c1-d625-4237-ba4e-8b1545d7df3d admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:12:38.876323 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 784/1578] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:12:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:12:38.877212 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680b4709-0b9f-4e11-b3e5-5f49f84a9406 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:12:38.887505 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680b4709-0b9f-4e11-b3e5-5f49f84a9406 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:12:38.892234 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b6280d-96fc-463c-8f2b-cb2eaa8d716b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:12:38.892830 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b6280d-96fc-463c-8f2b-cb2eaa8d716b admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:38.893108 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b6280d-96fc-463c-8f2b-cb2eaa8d716b admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:38.893374 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b6280d-96fc-463c-8f2b-cb2eaa8d716b admin admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:38.893992 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b6280d-96fc-463c-8f2b-cb2eaa8d716b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:12:38.895027 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: WARNING py.warnings [None req-65b6280d-96fc-463c-8f2b-cb2eaa8d716b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 12:12:38.895027 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: warnings.warn(msg) Sep 09 12:12:38.895027 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: Sep 09 12:12:38.899037 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 795/1579] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 9 12:12:38 2022] POST /identity/v3/projects => generated 389 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:12:38.899187 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b6280d-96fc-463c-8f2b-cb2eaa8d716b admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:12:38.899297 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28317c1-d625-4237-ba4e-8b1545d7df3d admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:12:38.921429 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 785/1580] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:12:38 2022] POST /identity/v3/users => generated 284 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:12:38.921429 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b6280d-96fc-463c-8f2b-cb2eaa8d716b admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:12:38.928858 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b6280d-96fc-463c-8f2b-cb2eaa8d716b admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:12:38.948636 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:12:38.962179 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28317c1-d625-4237-ba4e-8b1545d7df3d admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:12:38.967873 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6656699-5867-47b5-9057-92dc8bde6b3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:12:38.968649 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6656699-5867-47b5-9057-92dc8bde6b3b admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:38.969008 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6656699-5867-47b5-9057-92dc8bde6b3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:38.969355 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6656699-5867-47b5-9057-92dc8bde6b3b admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:38.970680 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6656699-5867-47b5-9057-92dc8bde6b3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:12:38.972511 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6656699-5867-47b5-9057-92dc8bde6b3b admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:12:38.980007 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:12:38.985799 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 796/1581] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:12:38 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:12:38.985799 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6656699-5867-47b5-9057-92dc8bde6b3b admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:12:38.993217 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6656699-5867-47b5-9057-92dc8bde6b3b admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:12:38.994052 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcc55db-1482-4991-821a-d285ed1d5abc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:12:38.994698 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcc55db-1482-4991-821a-d285ed1d5abc admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:38.995007 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcc55db-1482-4991-821a-d285ed1d5abc admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:38.995277 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcc55db-1482-4991-821a-d285ed1d5abc admin admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:38.996761 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcc55db-1482-4991-821a-d285ed1d5abc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:12:38.998692 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcc55db-1482-4991-821a-d285ed1d5abc admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:12:38.999041 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59187ab7-a344-475a-94e1-5eb48aff7058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:12:38.999631 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59187ab7-a344-475a-94e1-5eb48aff7058 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:38.999935 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59187ab7-a344-475a-94e1-5eb48aff7058 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:39.000232 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59187ab7-a344-475a-94e1-5eb48aff7058 admin admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:39.000847 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59187ab7-a344-475a-94e1-5eb48aff7058 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:12:39.002742 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59187ab7-a344-475a-94e1-5eb48aff7058 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:12:39.021463 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 786/1582] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:12:38 2022] POST /identity/v3/users => generated 292 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:12:39.022274 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcc55db-1482-4991-821a-d285ed1d5abc admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:12:39.030823 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 797/1583] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:12:38 2022] POST /identity/v3/users => generated 297 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:12:39.030823 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59187ab7-a344-475a-94e1-5eb48aff7058 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:12:39.034434 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcc55db-1482-4991-821a-d285ed1d5abc admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:12:39.038176 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59187ab7-a344-475a-94e1-5eb48aff7058 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:12:39.039179 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fef018b-fdfc-4b0a-95f0-96e7c0be811f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:12:39.039781 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fef018b-fdfc-4b0a-95f0-96e7c0be811f admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:39.040062 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fef018b-fdfc-4b0a-95f0-96e7c0be811f admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:39.040362 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fef018b-fdfc-4b0a-95f0-96e7c0be811f admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:39.040999 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fef018b-fdfc-4b0a-95f0-96e7c0be811f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:12:39.042647 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fef018b-fdfc-4b0a-95f0-96e7c0be811f admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:12:39.042776 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82eee374-761f-4893-9893-e194734431c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:12:39.043304 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eee374-761f-4893-9893-e194734431c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:39.043583 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eee374-761f-4893-9893-e194734431c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:39.043878 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eee374-761f-4893-9893-e194734431c5 admin admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:39.044494 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82eee374-761f-4893-9893-e194734431c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:12:39.046387 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82eee374-761f-4893-9893-e194734431c5 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:12:39.054675 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 787/1584] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:12:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:12:39.056098 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fef018b-fdfc-4b0a-95f0-96e7c0be811f admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:12:39.072833 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 798/1585] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:12:39 2022] POST /identity/v3/users => generated 284 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:12:39.073464 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82eee374-761f-4893-9893-e194734431c5 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:12:39.086871 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82eee374-761f-4893-9893-e194734431c5 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:12:39.092106 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af68a7c3-f358-41d0-8107-3ea937454d3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:12:39.093515 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af68a7c3-f358-41d0-8107-3ea937454d3a admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:39.093794 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af68a7c3-f358-41d0-8107-3ea937454d3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:39.094196 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af68a7c3-f358-41d0-8107-3ea937454d3a admin admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:39.094837 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af68a7c3-f358-41d0-8107-3ea937454d3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:12:39.096794 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af68a7c3-f358-41d0-8107-3ea937454d3a admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:12:39.113796 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fef018b-fdfc-4b0a-95f0-96e7c0be811f admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:12:39.117623 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10080279-2b09-438a-86f9-93dbc2ffac2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:12:39.118230 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10080279-2b09-438a-86f9-93dbc2ffac2a service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:39.118496 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10080279-2b09-438a-86f9-93dbc2ffac2a service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:39.118758 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10080279-2b09-438a-86f9-93dbc2ffac2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:39.122168 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 799/1586] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:12:39 2022] POST /identity/v3/users => generated 295 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:12:39.123146 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af68a7c3-f358-41d0-8107-3ea937454d3a admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:12:39.129175 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10080279-2b09-438a-86f9-93dbc2ffac2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:12:39.130763 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10080279-2b09-438a-86f9-93dbc2ffac2a service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:12:39.133578 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af68a7c3-f358-41d0-8107-3ea937454d3a admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:12:39.138701 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d745f0e-ab9b-403a-b135-9912e75e9826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:12:39.139315 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d745f0e-ab9b-403a-b135-9912e75e9826 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:39.139590 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d745f0e-ab9b-403a-b135-9912e75e9826 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:39.139858 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d745f0e-ab9b-403a-b135-9912e75e9826 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:39.140486 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d745f0e-ab9b-403a-b135-9912e75e9826 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:12:39.141949 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 788/1587] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:12:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:12:39.142177 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d745f0e-ab9b-403a-b135-9912e75e9826 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:12:39.142589 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10080279-2b09-438a-86f9-93dbc2ffac2a service neutron] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:12:39.151993 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10080279-2b09-438a-86f9-93dbc2ffac2a service neutron] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:12:39.153365 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 800/1588] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:12:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:12:39.153867 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d745f0e-ab9b-403a-b135-9912e75e9826 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:12:39.162579 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d58ebe-09dc-434c-ae91-28f9156b8c61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:12:39.163600 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d58ebe-09dc-434c-ae91-28f9156b8c61 admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:39.164103 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d58ebe-09dc-434c-ae91-28f9156b8c61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:39.164389 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d745f0e-ab9b-403a-b135-9912e75e9826 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:12:39.164713 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d58ebe-09dc-434c-ae91-28f9156b8c61 admin admin] PATH_INFO: `/v3/projects/433140f4c010464fa04a5622b26bc3ab/users/ab948442f326433c8e9f5f90314627aa/roles/eecd2f160dd844f5906f3d345780c501` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:39.169392 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a214626-1343-476c-9622-686abc553bd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:12:39.170048 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a214626-1343-476c-9622-686abc553bd2 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:39.170317 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a214626-1343-476c-9622-686abc553bd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:39.170553 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a214626-1343-476c-9622-686abc553bd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:39.171316 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a214626-1343-476c-9622-686abc553bd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:12:39.172986 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a214626-1343-476c-9622-686abc553bd2 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:12:39.183942 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d58ebe-09dc-434c-ae91-28f9156b8c61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=433140f4c010464fa04a5622b26bc3ab, user_id=ab948442f326433c8e9f5f90314627aa, role_id=eecd2f160dd844f5906f3d345780c501)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:12:39.186361 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 801/1589] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:12:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:12:39.187045 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a214626-1343-476c-9622-686abc553bd2 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:12:39.187304 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d58ebe-09dc-434c-ae91-28f9156b8c61 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:12:39.197441 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a214626-1343-476c-9622-686abc553bd2 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:12:39.206512 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 789/1590] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:12:39 2022] PUT /identity/v3/projects/433140f4c010464fa04a5622b26bc3ab/users/ab948442f326433c8e9f5f90314627aa/roles/eecd2f160dd844f5906f3d345780c501 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:12:39.207331 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d58ebe-09dc-434c-ae91-28f9156b8c61 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:12:39.269151 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ef51ed-1da9-4227-b202-cbfb91b83fad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:12:39.269776 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ef51ed-1da9-4227-b202-cbfb91b83fad admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:39.270057 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ef51ed-1da9-4227-b202-cbfb91b83fad admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:39.270327 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ef51ed-1da9-4227-b202-cbfb91b83fad admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:39.271004 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ef51ed-1da9-4227-b202-cbfb91b83fad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:12:39.272608 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ef51ed-1da9-4227-b202-cbfb91b83fad admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:12:39.276870 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d58ebe-09dc-434c-ae91-28f9156b8c61 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:12:39.282356 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eb89a8-69ff-4eb1-a71c-bbd3857f1ca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:12:39.283156 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 802/1591] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:12:39 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:12:39.283953 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ef51ed-1da9-4227-b202-cbfb91b83fad admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:12:39.286280 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eb89a8-69ff-4eb1-a71c-bbd3857f1ca0 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:39.286621 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eb89a8-69ff-4eb1-a71c-bbd3857f1ca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:39.286846 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eb89a8-69ff-4eb1-a71c-bbd3857f1ca0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:39.287559 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20eb89a8-69ff-4eb1-a71c-bbd3857f1ca0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:12:39.289250 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20eb89a8-69ff-4eb1-a71c-bbd3857f1ca0 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:12:39.295851 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ef51ed-1da9-4227-b202-cbfb91b83fad admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:12:39.300574 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56c05bd-a38d-460c-8bd6-8f44e645f332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:12:39.301185 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56c05bd-a38d-460c-8bd6-8f44e645f332 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:39.301753 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56c05bd-a38d-460c-8bd6-8f44e645f332 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:39.301882 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 790/1592] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:12:39 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:12:39.301968 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56c05bd-a38d-460c-8bd6-8f44e645f332 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:39.302520 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56c05bd-a38d-460c-8bd6-8f44e645f332 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:12:39.304176 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56c05bd-a38d-460c-8bd6-8f44e645f332 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:12:39.305611 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eb89a8-69ff-4eb1-a71c-bbd3857f1ca0 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:12:39.319608 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 803/1593] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:12:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:12:39.319608 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56c05bd-a38d-460c-8bd6-8f44e645f332 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:12:39.319897 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eb89a8-69ff-4eb1-a71c-bbd3857f1ca0 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:12:39.325334 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e49820-31f1-4ef4-9883-e3ae9dc073f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:12:39.325933 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e49820-31f1-4ef4-9883-e3ae9dc073f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:39.326278 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e49820-31f1-4ef4-9883-e3ae9dc073f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:39.326602 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e49820-31f1-4ef4-9883-e3ae9dc073f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:39.327602 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e49820-31f1-4ef4-9883-e3ae9dc073f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:12:39.329133 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e49820-31f1-4ef4-9883-e3ae9dc073f5 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:12:39.333295 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56c05bd-a38d-460c-8bd6-8f44e645f332 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:12:39.346119 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 791/1594] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:12:39 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:12:39.347425 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e49820-31f1-4ef4-9883-e3ae9dc073f5 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:12:39.349363 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66d708f-54d0-45bc-ac63-8592858340f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:12:39.350381 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66d708f-54d0-45bc-ac63-8592858340f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:39.350866 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66d708f-54d0-45bc-ac63-8592858340f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:39.351407 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66d708f-54d0-45bc-ac63-8592858340f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:39.352388 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66d708f-54d0-45bc-ac63-8592858340f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:12:39.356429 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66d708f-54d0-45bc-ac63-8592858340f3 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:12:39.360032 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e49820-31f1-4ef4-9883-e3ae9dc073f5 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:12:39.365887 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e4e793-d27a-4148-8a7f-70460952d0a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:12:39.366632 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e4e793-d27a-4148-8a7f-70460952d0a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:39.367048 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e4e793-d27a-4148-8a7f-70460952d0a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:39.367421 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e4e793-d27a-4148-8a7f-70460952d0a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:39.368168 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e4e793-d27a-4148-8a7f-70460952d0a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:12:39.369550 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 804/1595] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:12:39 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:12:39.370133 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e4e793-d27a-4148-8a7f-70460952d0a9 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:12:39.375348 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064483cc-54c2-4845-b80c-969c645a1673 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:39.375626 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064483cc-54c2-4845-b80c-969c645a1673 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:39.375892 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064483cc-54c2-4845-b80c-969c645a1673 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:39.383970 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 792/1596] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:12:39 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:12:39.384898 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e4e793-d27a-4148-8a7f-70460952d0a9 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:12:39.395700 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e4e793-d27a-4148-8a7f-70460952d0a9 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:12:39.398663 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-064483cc-54c2-4845-b80c-969c645a1673 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:12:39.399891 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-064483cc-54c2-4845-b80c-969c645a1673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:12:39.400910 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93eafe65-3834-44bf-86c9-5360aad0bb67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:12:39.401610 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eafe65-3834-44bf-86c9-5360aad0bb67 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:39.402073 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eafe65-3834-44bf-86c9-5360aad0bb67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:39.402669 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eafe65-3834-44bf-86c9-5360aad0bb67 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:39.403419 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93eafe65-3834-44bf-86c9-5360aad0bb67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:12:39.405088 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93eafe65-3834-44bf-86c9-5360aad0bb67 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:12:39.412489 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 805/1597] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:12:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:12:39.412489 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064483cc-54c2-4845-b80c-969c645a1673 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:12:39.421800 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 793/1598] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:12:39 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:12:39.421800 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93eafe65-3834-44bf-86c9-5360aad0bb67 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:12:39.427383 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064483cc-54c2-4845-b80c-969c645a1673 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:12:39.432719 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93eafe65-3834-44bf-86c9-5360aad0bb67 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:12:39.433109 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373febba-32ed-4412-8e0a-90eb20aa9728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:12:39.434461 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373febba-32ed-4412-8e0a-90eb20aa9728 admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:39.434741 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373febba-32ed-4412-8e0a-90eb20aa9728 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:39.435065 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373febba-32ed-4412-8e0a-90eb20aa9728 admin admin] PATH_INFO: `/v3/projects/8d9e072a5a2c432e946180291b81e1b0/users/6517ef70f96b4e089579b3f4cc4a1e91/roles/eecd2f160dd844f5906f3d345780c501` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:39.438143 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5586c3-8bb0-4ef1-b6fd-97339931c573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:12:39.439157 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5586c3-8bb0-4ef1-b6fd-97339931c573 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:39.439157 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5586c3-8bb0-4ef1-b6fd-97339931c573 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:39.439384 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5586c3-8bb0-4ef1-b6fd-97339931c573 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:39.440023 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5586c3-8bb0-4ef1-b6fd-97339931c573 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:12:39.441636 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5586c3-8bb0-4ef1-b6fd-97339931c573 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:12:39.452473 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373febba-32ed-4412-8e0a-90eb20aa9728 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d9e072a5a2c432e946180291b81e1b0, user_id=6517ef70f96b4e089579b3f4cc4a1e91, role_id=eecd2f160dd844f5906f3d345780c501)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:12:39.453930 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: WARNING py.warnings [None req-373febba-32ed-4412-8e0a-90eb20aa9728 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 12:12:39.453930 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: warnings.warn(msg) Sep 09 12:12:39.453930 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: Sep 09 12:12:39.454343 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 794/1599] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:12:39 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:12:39.455072 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5586c3-8bb0-4ef1-b6fd-97339931c573 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:12:39.460557 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373febba-32ed-4412-8e0a-90eb20aa9728 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:12:39.460913 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:12:39.468720 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5586c3-8bb0-4ef1-b6fd-97339931c573 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:12:39.474820 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5128945b-9b20-49b7-9c44-cd60017ac907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:12:39.475458 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5128945b-9b20-49b7-9c44-cd60017ac907 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:39.475789 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5128945b-9b20-49b7-9c44-cd60017ac907 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:39.476066 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5128945b-9b20-49b7-9c44-cd60017ac907 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:39.476690 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5128945b-9b20-49b7-9c44-cd60017ac907 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:12:39.489129 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5128945b-9b20-49b7-9c44-cd60017ac907 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:12:39.489365 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 806/1600] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:12:39 2022] PUT /identity/v3/projects/8d9e072a5a2c432e946180291b81e1b0/users/6517ef70f96b4e089579b3f4cc4a1e91/roles/eecd2f160dd844f5906f3d345780c501 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:12:39.489365 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373febba-32ed-4412-8e0a-90eb20aa9728 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:12:39.491612 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 795/1601] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:12:39 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:12:39.492466 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5128945b-9b20-49b7-9c44-cd60017ac907 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:12:39.506835 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5128945b-9b20-49b7-9c44-cd60017ac907 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:12:39.525851 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:12:39.587139 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b975133f-2c06-43c3-80ec-00e5fdfa3900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:12:39.590574 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b975133f-2c06-43c3-80ec-00e5fdfa3900 admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:39.591333 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b975133f-2c06-43c3-80ec-00e5fdfa3900 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:39.591608 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b975133f-2c06-43c3-80ec-00e5fdfa3900 admin admin] PATH_INFO: `/v3/projects/5cc0bd37a3024948b849e396014778e9/users/d3e5d2cb984744b3acd4e9445fe7e248/roles/eecd2f160dd844f5906f3d345780c501` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:39.594114 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373febba-32ed-4412-8e0a-90eb20aa9728 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:12:39.599141 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc8b8af-3b85-4541-a6db-e332c63bd528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:12:39.600100 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc8b8af-3b85-4541-a6db-e332c63bd528 admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:39.600579 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc8b8af-3b85-4541-a6db-e332c63bd528 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:39.600909 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc8b8af-3b85-4541-a6db-e332c63bd528 admin admin] PATH_INFO: `/v3/projects/9eaebfb93fb4475199e5cd07dba0be64/users/eac115bcbe744580ab33480c53ff95af/roles/eecd2f160dd844f5906f3d345780c501` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:39.608596 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b975133f-2c06-43c3-80ec-00e5fdfa3900 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cc0bd37a3024948b849e396014778e9, user_id=d3e5d2cb984744b3acd4e9445fe7e248, role_id=eecd2f160dd844f5906f3d345780c501)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:12:39.611898 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b975133f-2c06-43c3-80ec-00e5fdfa3900 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:12:39.620361 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc8b8af-3b85-4541-a6db-e332c63bd528 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9eaebfb93fb4475199e5cd07dba0be64, user_id=eac115bcbe744580ab33480c53ff95af, role_id=eecd2f160dd844f5906f3d345780c501)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:12:39.625987 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc8b8af-3b85-4541-a6db-e332c63bd528 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:12:39.630948 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 796/1602] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:12:39 2022] PUT /identity/v3/projects/5cc0bd37a3024948b849e396014778e9/users/d3e5d2cb984744b3acd4e9445fe7e248/roles/eecd2f160dd844f5906f3d345780c501 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:12:39.630948 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b975133f-2c06-43c3-80ec-00e5fdfa3900 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:12:39.656280 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 807/1603] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:12:39 2022] PUT /identity/v3/projects/9eaebfb93fb4475199e5cd07dba0be64/users/eac115bcbe744580ab33480c53ff95af/roles/eecd2f160dd844f5906f3d345780c501 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:12:39.657342 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc8b8af-3b85-4541-a6db-e332c63bd528 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:12:39.709458 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b975133f-2c06-43c3-80ec-00e5fdfa3900 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:12:39.732089 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc8b8af-3b85-4541-a6db-e332c63bd528 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:12:39.737225 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f819d4f-6f75-4825-8590-c5e7d20c91b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:12:39.738352 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f819d4f-6f75-4825-8590-c5e7d20c91b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:39.738744 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f819d4f-6f75-4825-8590-c5e7d20c91b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:39.739033 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f819d4f-6f75-4825-8590-c5e7d20c91b0 admin admin] PATH_INFO: `/v3/projects/edbc1c04ea364f3bb335fd4ad0731d1b/users/31411178e5c342b7bb14a21dec62f540/roles/eecd2f160dd844f5906f3d345780c501` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:39.756282 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f819d4f-6f75-4825-8590-c5e7d20c91b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=edbc1c04ea364f3bb335fd4ad0731d1b, user_id=31411178e5c342b7bb14a21dec62f540, role_id=eecd2f160dd844f5906f3d345780c501)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:12:39.759007 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f819d4f-6f75-4825-8590-c5e7d20c91b0 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:12:39.765943 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47e1a40-51fa-4a3d-b680-14576fc71210 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:12:39.766622 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47e1a40-51fa-4a3d-b680-14576fc71210 service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:39.766991 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47e1a40-51fa-4a3d-b680-14576fc71210 service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:39.767386 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47e1a40-51fa-4a3d-b680-14576fc71210 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:39.775215 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 808/1604] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:12:39 2022] PUT /identity/v3/projects/edbc1c04ea364f3bb335fd4ad0731d1b/users/31411178e5c342b7bb14a21dec62f540/roles/eecd2f160dd844f5906f3d345780c501 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:12:39.778635 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47e1a40-51fa-4a3d-b680-14576fc71210 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:12:39.779942 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f819d4f-6f75-4825-8590-c5e7d20c91b0 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:12:39.780224 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47e1a40-51fa-4a3d-b680-14576fc71210 service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:12:39.790661 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 797/1605] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:12:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:12:39.792183 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc09ba4-a85c-4e8b-9362-2fffcafddfd7 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:39.792831 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc09ba4-a85c-4e8b-9362-2fffcafddfd7 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:39.793203 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc09ba4-a85c-4e8b-9362-2fffcafddfd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:39.826008 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-bbc09ba4-a85c-4e8b-9362-2fffcafddfd7 None None] MFA Rules not processed for user `6517ef70f96b4e089579b3f4cc4a1e91`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:12:39.827594 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-bbc09ba4-a85c-4e8b-9362-2fffcafddfd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:12:39.857722 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f819d4f-6f75-4825-8590-c5e7d20c91b0 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:12:39.862859 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b06b005-e29b-414c-96a5-073dcb156b97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:12:39.863597 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:12:39.863849 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b06b005-e29b-414c-96a5-073dcb156b97 admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:39.864204 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b06b005-e29b-414c-96a5-073dcb156b97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:39.864545 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b06b005-e29b-414c-96a5-073dcb156b97 admin admin] PATH_INFO: `/v3/projects/977164fbbeac4040a6d2a7c965ea1d9a/users/c18cf3d809c24930a4b08930a2835a7f/roles/eecd2f160dd844f5906f3d345780c501` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:39.883402 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b06b005-e29b-414c-96a5-073dcb156b97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=977164fbbeac4040a6d2a7c965ea1d9a, user_id=c18cf3d809c24930a4b08930a2835a7f, role_id=eecd2f160dd844f5906f3d345780c501)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:12:39.887077 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b06b005-e29b-414c-96a5-073dcb156b97 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:12:39.907429 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 809/1606] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:12:39 2022] PUT /identity/v3/projects/977164fbbeac4040a6d2a7c965ea1d9a/users/c18cf3d809c24930a4b08930a2835a7f/roles/eecd2f160dd844f5906f3d345780c501 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:12:39.907429 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b06b005-e29b-414c-96a5-073dcb156b97 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:12:39.921251 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 798/1607] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:12:39 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:12:39.922720 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb558e3-599b-4c09-925d-b1568f25f992 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:39.923767 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb558e3-599b-4c09-925d-b1568f25f992 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:39.924080 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb558e3-599b-4c09-925d-b1568f25f992 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:39.926007 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:12:39.963920 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-bbb558e3-599b-4c09-925d-b1568f25f992 None None] MFA Rules not processed for user `d3e5d2cb984744b3acd4e9445fe7e248`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:12:39.966378 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-bbb558e3-599b-4c09-925d-b1568f25f992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:12:40.031285 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b06b005-e29b-414c-96a5-073dcb156b97 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:12:40.056035 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a13461-63d4-438f-b554-4c16b1181847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:12:40.056724 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a13461-63d4-438f-b554-4c16b1181847 admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:40.057391 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a13461-63d4-438f-b554-4c16b1181847 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:40.057670 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a13461-63d4-438f-b554-4c16b1181847 admin admin] PATH_INFO: `/v3/projects/f2f72b6ae8d3482e935d3375bc999732/users/b9a5c9fb3f664ae097f0c3ce6f6c0764/roles/eecd2f160dd844f5906f3d345780c501` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:40.077108 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 799/1608] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:12:39 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:12:40.083658 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c275e3-da0f-42d2-9eb3-3c9840746d71 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:40.083969 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c275e3-da0f-42d2-9eb3-3c9840746d71 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:40.084272 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c275e3-da0f-42d2-9eb3-3c9840746d71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:40.085373 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a13461-63d4-438f-b554-4c16b1181847 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2f72b6ae8d3482e935d3375bc999732, user_id=b9a5c9fb3f664ae097f0c3ce6f6c0764, role_id=eecd2f160dd844f5906f3d345780c501)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:12:40.088405 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a13461-63d4-438f-b554-4c16b1181847 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:12:40.118504 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 810/1609] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:12:39 2022] PUT /identity/v3/projects/f2f72b6ae8d3482e935d3375bc999732/users/b9a5c9fb3f664ae097f0c3ce6f6c0764/roles/eecd2f160dd844f5906f3d345780c501 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:12:40.118504 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019a5fa4-6077-40cc-ad3a-bff362168919 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:40.118504 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019a5fa4-6077-40cc-ad3a-bff362168919 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:40.118504 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019a5fa4-6077-40cc-ad3a-bff362168919 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:40.134359 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-e6c275e3-da0f-42d2-9eb3-3c9840746d71 None None] MFA Rules not processed for user `eac115bcbe744580ab33480c53ff95af`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:12:40.135704 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-e6c275e3-da0f-42d2-9eb3-3c9840746d71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:12:40.169987 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-019a5fa4-6077-40cc-ad3a-bff362168919 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:12:40.171386 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-019a5fa4-6077-40cc-ad3a-bff362168919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:12:40.249809 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 800/1610] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:12:40 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:12:40.251459 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76d910d-afe5-414c-a2ff-b13f94c17626 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:40.251596 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76d910d-afe5-414c-a2ff-b13f94c17626 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:40.251901 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76d910d-afe5-414c-a2ff-b13f94c17626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:40.299108 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-a76d910d-afe5-414c-a2ff-b13f94c17626 None None] MFA Rules not processed for user `c18cf3d809c24930a4b08930a2835a7f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:12:40.300559 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-a76d910d-afe5-414c-a2ff-b13f94c17626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:12:40.321448 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 811/1611] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:12:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:12:40.326169 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019a5fa4-6077-40cc-ad3a-bff362168919 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:12:40.337042 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019a5fa4-6077-40cc-ad3a-bff362168919 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:12:40.342877 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f9f449-21af-4601-9564-0faf19ce4efa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:12:40.343645 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f9f449-21af-4601-9564-0faf19ce4efa admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:40.344038 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f9f449-21af-4601-9564-0faf19ce4efa admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:40.344804 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f9f449-21af-4601-9564-0faf19ce4efa admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:40.345809 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f9f449-21af-4601-9564-0faf19ce4efa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:12:40.348392 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f9f449-21af-4601-9564-0faf19ce4efa admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:12:40.374685 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:12:40.398089 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 812/1612] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 9 12:12:40 2022] POST /identity/v3/projects => generated 371 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:12:40.399401 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64062a67-5ba9-4274-bc37-827fbacdeb65 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:40.399685 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64062a67-5ba9-4274-bc37-827fbacdeb65 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:40.399958 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64062a67-5ba9-4274-bc37-827fbacdeb65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:40.437932 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-64062a67-5ba9-4274-bc37-827fbacdeb65 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:12:40.439201 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-64062a67-5ba9-4274-bc37-827fbacdeb65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:12:40.446268 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 801/1613] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:12:40 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:12:40.447778 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4595039-7e50-4130-b655-373fbaa748dd None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:40.448033 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4595039-7e50-4130-b655-373fbaa748dd None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:40.448304 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4595039-7e50-4130-b655-373fbaa748dd None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:40.449746 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 802/1614] 10.222.0.49 () {54 vars in 962 bytes} [Fri Sep 9 12:12:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:12:40.459659 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff7a9c5-762f-4c85-ae0a-b60ad2fb58be None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:40.459939 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff7a9c5-762f-4c85-ae0a-b60ad2fb58be None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:40.460512 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff7a9c5-762f-4c85-ae0a-b60ad2fb58be None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:40.470628 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 803/1615] 10.222.0.49 () {54 vars in 962 bytes} [Fri Sep 9 12:12:40 2022] GET /identity => generated 270 bytes in 20 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:12:40.472642 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff7a9c5-762f-4c85-ae0a-b60ad2fb58be None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:12:40.561413 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 813/1616] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:12:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:12:40.562547 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64062a67-5ba9-4274-bc37-827fbacdeb65 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:12:40.575761 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64062a67-5ba9-4274-bc37-827fbacdeb65 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:12:40.582717 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc26a7d9-226f-44bb-b14e-3120aca6a5f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:12:40.583824 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc26a7d9-226f-44bb-b14e-3120aca6a5f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:40.584277 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc26a7d9-226f-44bb-b14e-3120aca6a5f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:40.584786 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc26a7d9-226f-44bb-b14e-3120aca6a5f3 admin admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:40.585952 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc26a7d9-226f-44bb-b14e-3120aca6a5f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:12:40.589287 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc26a7d9-226f-44bb-b14e-3120aca6a5f3 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:12:40.599374 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:12:40.620700 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 814/1617] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:12:40 2022] POST /identity/v3/users => generated 292 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:12:40.623029 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d00bb0c-f57f-48b2-b27b-4d58eee0f53a None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:40.623525 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d00bb0c-f57f-48b2-b27b-4d58eee0f53a None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:40.623981 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d00bb0c-f57f-48b2-b27b-4d58eee0f53a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:40.624740 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff7a9c5-762f-4c85-ae0a-b60ad2fb58be None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:12:40.636123 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59368499-c045-475c-bf00-41ff4852472a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:12:40.637004 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59368499-c045-475c-bf00-41ff4852472a service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:40.637454 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59368499-c045-475c-bf00-41ff4852472a service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:40.637726 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59368499-c045-475c-bf00-41ff4852472a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:40.671175 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59368499-c045-475c-bf00-41ff4852472a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:12:40.672776 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59368499-c045-475c-bf00-41ff4852472a service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:12:40.684235 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 804/1618] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:12:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:12:40.685925 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59368499-c045-475c-bf00-41ff4852472a service neutron] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:12:40.698654 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-9d00bb0c-f57f-48b2-b27b-4d58eee0f53a None None] MFA Rules not processed for user `70bac169d5cd43e1926f310344ac11fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:12:40.704596 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-9d00bb0c-f57f-48b2-b27b-4d58eee0f53a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:12:40.818042 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59368499-c045-475c-bf00-41ff4852472a service neutron] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:12:40.826249 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 815/1619] 10.222.0.49 () {58 vars in 1059 bytes} [Fri Sep 9 12:12:40 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:12:40.827234 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a533c569-5a5f-4c91-9b91-796cc5558276 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535c258100df40b58a5ee44b2c21d621', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:12:40.827894 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a533c569-5a5f-4c91-9b91-796cc5558276 service cinder] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:40.828019 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6318d7-60f3-492b-ade4-b76b5633614e None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:40.828170 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a533c569-5a5f-4c91-9b91-796cc5558276 service cinder] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:40.828347 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6318d7-60f3-492b-ade4-b76b5633614e None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:40.828466 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a533c569-5a5f-4c91-9b91-796cc5558276 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:40.828630 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6318d7-60f3-492b-ade4-b76b5633614e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:40.852974 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-cb6318d7-60f3-492b-ade4-b76b5633614e None None] MFA Rules not processed for user `70bac169d5cd43e1926f310344ac11fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:12:40.855742 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-cb6318d7-60f3-492b-ade4-b76b5633614e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:12:40.872027 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 816/1620] 10.222.0.49 () {58 vars in 1059 bytes} [Fri Sep 9 12:12:40 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:12:40.873166 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6318d7-60f3-492b-ade4-b76b5633614e None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:12:40.890454 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6318d7-60f3-492b-ade4-b76b5633614e None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:12:40.895624 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4cd77d-9c91-4593-ae26-bd99a254040d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:12:40.896876 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4cd77d-9c91-4593-ae26-bd99a254040d admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:40.897259 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4cd77d-9c91-4593-ae26-bd99a254040d admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:40.897680 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4cd77d-9c91-4593-ae26-bd99a254040d admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:40.898663 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4cd77d-9c91-4593-ae26-bd99a254040d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:12:40.900918 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4cd77d-9c91-4593-ae26-bd99a254040d admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:12:40.939054 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 817/1621] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:12:40 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:12:40.941823 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4cd77d-9c91-4593-ae26-bd99a254040d admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:12:40.976954 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4cd77d-9c91-4593-ae26-bd99a254040d admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:12:40.990692 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8242ce0-b8a6-4ff7-a2ae-5233f02b5280 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:12:40.991324 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8242ce0-b8a6-4ff7-a2ae-5233f02b5280 service neutron] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:40.991791 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8242ce0-b8a6-4ff7-a2ae-5233f02b5280 service neutron] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:40.993420 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8242ce0-b8a6-4ff7-a2ae-5233f02b5280 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:41.011151 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a533c569-5a5f-4c91-9b91-796cc5558276 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:12:41.011800 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a533c569-5a5f-4c91-9b91-796cc5558276 service cinder] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:12:41.025814 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8242ce0-b8a6-4ff7-a2ae-5233f02b5280 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:12:41.028041 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8242ce0-b8a6-4ff7-a2ae-5233f02b5280 service neutron] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:12:41.048933 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 805/1622] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:12:40 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:12:41.050226 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a533c569-5a5f-4c91-9b91-796cc5558276 service cinder] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:12:41.055716 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 818/1623] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:12:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:12:41.056764 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8242ce0-b8a6-4ff7-a2ae-5233f02b5280 service neutron] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:12:41.075390 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a533c569-5a5f-4c91-9b91-796cc5558276 service cinder] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:12:41.079665 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8242ce0-b8a6-4ff7-a2ae-5233f02b5280 service neutron] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:12:41.085778 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2847ebb-66b6-4b4c-8966-5d2ef28d3c55 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70bac169d5cd43e1926f310344ac11fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:12:41.086779 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2847ebb-66b6-4b4c-8966-5d2ef28d3c55 service nova] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:41.087326 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2847ebb-66b6-4b4c-8966-5d2ef28d3c55 service nova] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:41.087828 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2847ebb-66b6-4b4c-8966-5d2ef28d3c55 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:41.088834 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77a4a1b-e15a-461e-82e4-5b2ba6c1fa68 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70bac169d5cd43e1926f310344ac11fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:12:41.089503 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77a4a1b-e15a-461e-82e4-5b2ba6c1fa68 service nova] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:41.089852 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77a4a1b-e15a-461e-82e4-5b2ba6c1fa68 service nova] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:41.090284 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77a4a1b-e15a-461e-82e4-5b2ba6c1fa68 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:41.141506 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:12:41.196008 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2847ebb-66b6-4b4c-8966-5d2ef28d3c55 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:12:41.198176 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2847ebb-66b6-4b4c-8966-5d2ef28d3c55 service nova] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:12:41.222138 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 806/1624] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:12:41 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:12:41.223058 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2847ebb-66b6-4b4c-8966-5d2ef28d3c55 service nova] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:12:41.237063 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2847ebb-66b6-4b4c-8966-5d2ef28d3c55 service nova] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:12:41.254403 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5947058-f9eb-45c1-94b3-fb4f7949a497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:12:41.256477 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5947058-f9eb-45c1-94b3-fb4f7949a497 admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:41.256901 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5947058-f9eb-45c1-94b3-fb4f7949a497 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:41.257293 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5947058-f9eb-45c1-94b3-fb4f7949a497 admin admin] PATH_INFO: `/v3/projects/1a295491c5374ce9a47d7970bc83b400/users/1b62daa8af5144499533f893cda3f57e/roles/3f2ca33902a04b6dbc2ef7d06d7289d3` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:41.297201 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77a4a1b-e15a-461e-82e4-5b2ba6c1fa68 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:12:41.304304 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77a4a1b-e15a-461e-82e4-5b2ba6c1fa68 service nova] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:12:41.308848 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5947058-f9eb-45c1-94b3-fb4f7949a497 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a295491c5374ce9a47d7970bc83b400, user_id=1b62daa8af5144499533f893cda3f57e, role_id=3f2ca33902a04b6dbc2ef7d06d7289d3)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:12:41.312613 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5947058-f9eb-45c1-94b3-fb4f7949a497 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:12:41.380976 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 807/1625] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:12:41 2022] PUT /identity/v3/projects/1a295491c5374ce9a47d7970bc83b400/users/1b62daa8af5144499533f893cda3f57e/roles/3f2ca33902a04b6dbc2ef7d06d7289d3 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:12:41.381592 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 819/1626] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:12:41 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:12:41.381878 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5947058-f9eb-45c1-94b3-fb4f7949a497 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:12:41.515076 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5947058-f9eb-45c1-94b3-fb4f7949a497 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:12:41.537831 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842269cd-6c1c-45f5-9286-81cfce4a13d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:12:41.538612 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842269cd-6c1c-45f5-9286-81cfce4a13d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:41.538935 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842269cd-6c1c-45f5-9286-81cfce4a13d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:41.539207 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842269cd-6c1c-45f5-9286-81cfce4a13d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:41.539865 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842269cd-6c1c-45f5-9286-81cfce4a13d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:12:41.543007 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842269cd-6c1c-45f5-9286-81cfce4a13d6 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:12:41.577654 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 808/1627] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:12:41 2022] GET /identity/v3/roles => generated 1420 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:12:41.582640 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77a4a1b-e15a-461e-82e4-5b2ba6c1fa68 service nova] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:12:41.598075 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77a4a1b-e15a-461e-82e4-5b2ba6c1fa68 service nova] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:12:41.616936 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597aade7-d042-48c6-bdeb-42d62e16fefd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:12:41.617859 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597aade7-d042-48c6-bdeb-42d62e16fefd admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:41.617859 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597aade7-d042-48c6-bdeb-42d62e16fefd admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:41.618085 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597aade7-d042-48c6-bdeb-42d62e16fefd admin admin] PATH_INFO: `/v3/projects/1a295491c5374ce9a47d7970bc83b400/users/1b62daa8af5144499533f893cda3f57e/roles/3f2ca33902a04b6dbc2ef7d06d7289d3` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:41.631045 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597aade7-d042-48c6-bdeb-42d62e16fefd admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a295491c5374ce9a47d7970bc83b400, user_id=1b62daa8af5144499533f893cda3f57e, role_id=3f2ca33902a04b6dbc2ef7d06d7289d3)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:12:41.637769 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597aade7-d042-48c6-bdeb-42d62e16fefd admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:12:41.673823 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 820/1628] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:12:41 2022] PUT /identity/v3/projects/1a295491c5374ce9a47d7970bc83b400/users/1b62daa8af5144499533f893cda3f57e/roles/3f2ca33902a04b6dbc2ef7d06d7289d3 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:12:41.684867 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf385d90-ba95-4be5-b140-e29bccb19e36 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:41.685348 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf385d90-ba95-4be5-b140-e29bccb19e36 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:41.685443 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf385d90-ba95-4be5-b140-e29bccb19e36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:41.753126 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-bf385d90-ba95-4be5-b140-e29bccb19e36 None None] MFA Rules not processed for user `1b62daa8af5144499533f893cda3f57e`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:12:41.755489 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-bf385d90-ba95-4be5-b140-e29bccb19e36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:12:41.808109 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c05425-8316-49d2-b44d-1e430e9c1f46 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:41.808957 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c05425-8316-49d2-b44d-1e430e9c1f46 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:41.809253 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c05425-8316-49d2-b44d-1e430e9c1f46 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:41.810825 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 821/1629] 10.222.0.49 () {54 vars in 912 bytes} [Fri Sep 9 12:12:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 12:12:41.817033 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03f5084-6ae9-40ee-9c69-dcc67abccc0f None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:41.817381 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03f5084-6ae9-40ee-9c69-dcc67abccc0f None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:41.817642 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03f5084-6ae9-40ee-9c69-dcc67abccc0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:41.874856 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-e03f5084-6ae9-40ee-9c69-dcc67abccc0f None None] MFA Rules not processed for user `70bac169d5cd43e1926f310344ac11fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:12:41.879545 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-e03f5084-6ae9-40ee-9c69-dcc67abccc0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:12:41.927922 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:12:41.938554 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 809/1630] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:12:41 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:12:41.947812 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31550fd6-6f7a-49ba-87c9-0239c12bba64 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:41.949271 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31550fd6-6f7a-49ba-87c9-0239c12bba64 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:41.949563 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31550fd6-6f7a-49ba-87c9-0239c12bba64 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:41.955311 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 810/1631] 10.222.0.49 () {54 vars in 912 bytes} [Fri Sep 9 12:12:41 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:12:41.961041 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b725a5d7-5af7-42ef-953c-ee87141a296e None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:41.961324 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b725a5d7-5af7-42ef-953c-ee87141a296e None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:41.961594 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b725a5d7-5af7-42ef-953c-ee87141a296e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:42.012664 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-b725a5d7-5af7-42ef-953c-ee87141a296e None None] MFA Rules not processed for user `70bac169d5cd43e1926f310344ac11fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:12:42.013970 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-b725a5d7-5af7-42ef-953c-ee87141a296e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:12:42.037836 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 822/1632] 10.222.0.49 () {58 vars in 1009 bytes} [Fri Sep 9 12:12:41 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:12:42.039071 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03f5084-6ae9-40ee-9c69-dcc67abccc0f None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:12:42.063702 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:12:42.177848 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 811/1633] 10.222.0.49 () {58 vars in 1009 bytes} [Fri Sep 9 12:12:41 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:12:42.179328 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b725a5d7-5af7-42ef-953c-ee87141a296e None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:12:42.183517 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03f5084-6ae9-40ee-9c69-dcc67abccc0f None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:12:42.191298 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57f2433-5c0b-411f-961b-f9c403072dfb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535c258100df40b58a5ee44b2c21d621', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:12:42.193087 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57f2433-5c0b-411f-961b-f9c403072dfb service cinder] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:42.193371 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57f2433-5c0b-411f-961b-f9c403072dfb service cinder] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:42.193650 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57f2433-5c0b-411f-961b-f9c403072dfb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:42.204220 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b725a5d7-5af7-42ef-953c-ee87141a296e None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:12:42.213538 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7947f13c-a361-46a1-9fc2-01a61a0733f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535c258100df40b58a5ee44b2c21d621', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:12:42.215066 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7947f13c-a361-46a1-9fc2-01a61a0733f2 service cinder] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:42.215820 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7947f13c-a361-46a1-9fc2-01a61a0733f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:42.216366 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7947f13c-a361-46a1-9fc2-01a61a0733f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:42.240477 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7947f13c-a361-46a1-9fc2-01a61a0733f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:12:42.242187 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7947f13c-a361-46a1-9fc2-01a61a0733f2 service cinder] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:12:42.251438 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57f2433-5c0b-411f-961b-f9c403072dfb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:12:42.253085 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57f2433-5c0b-411f-961b-f9c403072dfb service cinder] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:12:42.264098 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 812/1634] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:12:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:12:42.283032 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 823/1635] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:12:42 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:12:42.644322 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7947f13c-a361-46a1-9fc2-01a61a0733f2 service cinder] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:12:42.656338 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7947f13c-a361-46a1-9fc2-01a61a0733f2 service cinder] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:12:42.661378 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7ba9a1-6793-4e66-8eaa-36814d6e769e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535c258100df40b58a5ee44b2c21d621', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:12:42.662238 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7ba9a1-6793-4e66-8eaa-36814d6e769e service cinder] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:42.662686 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7ba9a1-6793-4e66-8eaa-36814d6e769e service cinder] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:42.663653 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7ba9a1-6793-4e66-8eaa-36814d6e769e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:42.678279 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7ba9a1-6793-4e66-8eaa-36814d6e769e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:12:42.680344 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7ba9a1-6793-4e66-8eaa-36814d6e769e service cinder] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:12:42.700661 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 813/1636] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:12:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:12:43.123922 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57f2433-5c0b-411f-961b-f9c403072dfb service cinder] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:12:43.137826 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57f2433-5c0b-411f-961b-f9c403072dfb service cinder] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:12:43.154295 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42ff34c-b106-415c-a064-faf19e303b97 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535c258100df40b58a5ee44b2c21d621', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:12:43.155231 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42ff34c-b106-415c-a064-faf19e303b97 service cinder] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:43.155507 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42ff34c-b106-415c-a064-faf19e303b97 service cinder] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:43.155772 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42ff34c-b106-415c-a064-faf19e303b97 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:43.301012 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42ff34c-b106-415c-a064-faf19e303b97 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:12:43.302655 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42ff34c-b106-415c-a064-faf19e303b97 service cinder] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:12:43.333218 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 824/1637] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:12:43 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:12:48.147875 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7ba9a1-6793-4e66-8eaa-36814d6e769e service cinder] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:12:48.386584 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7ba9a1-6793-4e66-8eaa-36814d6e769e service cinder] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:12:48.423148 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818167be-0645-4348-9ce9-69f1a4000d62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:12:48.424179 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818167be-0645-4348-9ce9-69f1a4000d62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:48.424653 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818167be-0645-4348-9ce9-69f1a4000d62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:48.425132 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818167be-0645-4348-9ce9-69f1a4000d62 admin admin] PATH_INFO: `/v3/users/eac115bcbe744580ab33480c53ff95af` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:48.428584 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818167be-0645-4348-9ce9-69f1a4000d62 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eac115bcbe744580ab33480c53ff95af)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:12:48.430973 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818167be-0645-4348-9ce9-69f1a4000d62 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:12:48.580074 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-818167be-0645-4348-9ce9-69f1a4000d62 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eac115bcbe744580ab33480c53ff95af'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:12:48.643985 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-818167be-0645-4348-9ce9-69f1a4000d62 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eac115bcbe744580ab33480c53ff95af'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:12:48.683778 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-818167be-0645-4348-9ce9-69f1a4000d62 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eac115bcbe744580ab33480c53ff95af'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:12:48.717850 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-818167be-0645-4348-9ce9-69f1a4000d62 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eac115bcbe744580ab33480c53ff95af'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:12:48.720018 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-818167be-0645-4348-9ce9-69f1a4000d62 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eac115bcbe744580ab33480c53ff95af'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:12:48.728079 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 814/1638] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 9 12:12:48 2022] DELETE /identity/v3/users/eac115bcbe744580ab33480c53ff95af => generated 0 bytes in 581 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:12:48.733585 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5193a2fb-da35-4b23-8501-d1df7e8dae68 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:48.734105 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5193a2fb-da35-4b23-8501-d1df7e8dae68 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:48.734177 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5193a2fb-da35-4b23-8501-d1df7e8dae68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:48.807646 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-5193a2fb-da35-4b23-8501-d1df7e8dae68 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:12:48.810853 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-5193a2fb-da35-4b23-8501-d1df7e8dae68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:12:48.974712 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:12:49.037313 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 825/1639] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:12:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:12:49.066250 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818167be-0645-4348-9ce9-69f1a4000d62 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:12:49.069044 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-818167be-0645-4348-9ce9-69f1a4000d62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:12:49.111133 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:12:49.283814 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818167be-0645-4348-9ce9-69f1a4000d62 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:12:49.301598 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f86674-bae8-4fa8-855f-be2ccde5bde6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:12:49.302231 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f86674-bae8-4fa8-855f-be2ccde5bde6 service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:49.302506 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f86674-bae8-4fa8-855f-be2ccde5bde6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:49.302771 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f86674-bae8-4fa8-855f-be2ccde5bde6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:49.357234 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f86674-bae8-4fa8-855f-be2ccde5bde6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:12:49.368269 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f86674-bae8-4fa8-855f-be2ccde5bde6 service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:12:49.396844 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 815/1640] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:12:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:12:50.670686 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5193a2fb-da35-4b23-8501-d1df7e8dae68 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:12:50.671586 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cb6ff8-a65a-48de-926b-4697a658349d None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:50.672478 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cb6ff8-a65a-48de-926b-4697a658349d None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:50.673019 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cb6ff8-a65a-48de-926b-4697a658349d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:50.674678 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-5193a2fb-da35-4b23-8501-d1df7e8dae68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:12:50.720362 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5193a2fb-da35-4b23-8501-d1df7e8dae68 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:12:50.735636 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff6c4e2-a0b2-4a0f-8f15-050d341a6be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:12:50.736300 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff6c4e2-a0b2-4a0f-8f15-050d341a6be5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:50.736584 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff6c4e2-a0b2-4a0f-8f15-050d341a6be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:50.736862 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff6c4e2-a0b2-4a0f-8f15-050d341a6be5 admin admin] PATH_INFO: `/v3/projects/9eaebfb93fb4475199e5cd07dba0be64` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:50.738278 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff6c4e2-a0b2-4a0f-8f15-050d341a6be5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9eaebfb93fb4475199e5cd07dba0be64)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:12:50.740232 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff6c4e2-a0b2-4a0f-8f15-050d341a6be5 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:12:50.746854 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-02cb6ff8-a65a-48de-926b-4697a658349d None None] MFA Rules not processed for user `31411178e5c342b7bb14a21dec62f540`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:12:50.748930 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-02cb6ff8-a65a-48de-926b-4697a658349d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:12:50.893008 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-bff6c4e2-a0b2-4a0f-8f15-050d341a6be5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9eaebfb93fb4475199e5cd07dba0be64', 'request_id': 'req-bff6c4e2-a0b2-4a0f-8f15-050d341a6be5'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:12:50.903519 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 816/1641] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:12:50 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:12:50.915614 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a3c1b3-cf7a-4b14-9487-ee38f8fa81e6 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:50.918852 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a3c1b3-cf7a-4b14-9487-ee38f8fa81e6 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:50.919404 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a3c1b3-cf7a-4b14-9487-ee38f8fa81e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:50.955939 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-bff6c4e2-a0b2-4a0f-8f15-050d341a6be5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9eaebfb93fb4475199e5cd07dba0be64', 'request_id': 'req-bff6c4e2-a0b2-4a0f-8f15-050d341a6be5'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:12:50.972935 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-bff6c4e2-a0b2-4a0f-8f15-050d341a6be5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9eaebfb93fb4475199e5cd07dba0be64', 'request_id': 'req-bff6c4e2-a0b2-4a0f-8f15-050d341a6be5'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:12:50.986924 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-81a3c1b3-cf7a-4b14-9487-ee38f8fa81e6 None None] MFA Rules not processed for user `31411178e5c342b7bb14a21dec62f540`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:12:50.988605 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-81a3c1b3-cf7a-4b14-9487-ee38f8fa81e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:12:51.002410 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-bff6c4e2-a0b2-4a0f-8f15-050d341a6be5 admin admin] The token cache is being invalidate because project 9eaebfb93fb4475199e5cd07dba0be64 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81591) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:12:51.004214 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-bff6c4e2-a0b2-4a0f-8f15-050d341a6be5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:12:51.006672 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-bff6c4e2-a0b2-4a0f-8f15-050d341a6be5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:12:51.015889 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 826/1642] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 9 12:12:50 2022] DELETE /identity/v3/projects/9eaebfb93fb4475199e5cd07dba0be64 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:12:51.027271 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4383903-8713-461c-9f0c-47cd9c51b7ce None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:51.027909 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4383903-8713-461c-9f0c-47cd9c51b7ce None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:51.028296 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4383903-8713-461c-9f0c-47cd9c51b7ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:51.072371 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-c4383903-8713-461c-9f0c-47cd9c51b7ce None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:12:51.075054 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-c4383903-8713-461c-9f0c-47cd9c51b7ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:12:51.158947 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 817/1643] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:12:50 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:12:51.166124 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a3c1b3-cf7a-4b14-9487-ee38f8fa81e6 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:12:51.168415 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-81a3c1b3-cf7a-4b14-9487-ee38f8fa81e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:12:51.208248 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 827/1644] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:12:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:12:51.210016 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a258173d-0957-48c6-89a8-268a4a2d9316 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:51.210526 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a258173d-0957-48c6-89a8-268a4a2d9316 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:51.210802 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a258173d-0957-48c6-89a8-268a4a2d9316 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:51.245217 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-a258173d-0957-48c6-89a8-268a4a2d9316 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:12:51.246875 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-a258173d-0957-48c6-89a8-268a4a2d9316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:12:51.260021 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 828/1645] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:12:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:12:51.262028 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a258173d-0957-48c6-89a8-268a4a2d9316 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:12:51.275497 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a258173d-0957-48c6-89a8-268a4a2d9316 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:12:51.300330 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9f3c1b-7238-436e-a30b-f5cecba15d82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:12:51.300974 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9f3c1b-7238-436e-a30b-f5cecba15d82 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:51.301253 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9f3c1b-7238-436e-a30b-f5cecba15d82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:51.301540 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9f3c1b-7238-436e-a30b-f5cecba15d82 admin admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:51.306116 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:12:51.308818 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9f3c1b-7238-436e-a30b-f5cecba15d82 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:12:51.310710 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9f3c1b-7238-436e-a30b-f5cecba15d82 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:12:51.311898 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9f3c1b-7238-436e-a30b-f5cecba15d82 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:12:51.365698 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 829/1646] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 9 12:12:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:12:51.366647 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9f3c1b-7238-436e-a30b-f5cecba15d82 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:12:51.387171 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9f3c1b-7238-436e-a30b-f5cecba15d82 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:12:51.392705 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a3c1b3-cf7a-4b14-9487-ee38f8fa81e6 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:12:51.406173 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afe02ec-01de-4048-9120-d0d865fa9225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:12:51.406810 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afe02ec-01de-4048-9120-d0d865fa9225 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:51.412883 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0823bb0e-bb2f-4a3f-b4ef-291612d280b3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70bac169d5cd43e1926f310344ac11fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:12:51.413537 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0823bb0e-bb2f-4a3f-b4ef-291612d280b3 service nova] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:51.413819 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0823bb0e-bb2f-4a3f-b4ef-291612d280b3 service nova] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:51.414092 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0823bb0e-bb2f-4a3f-b4ef-291612d280b3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:51.414992 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afe02ec-01de-4048-9120-d0d865fa9225 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:51.415308 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afe02ec-01de-4048-9120-d0d865fa9225 admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:51.415983 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afe02ec-01de-4048-9120-d0d865fa9225 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:12:51.418607 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afe02ec-01de-4048-9120-d0d865fa9225 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:12:51.423754 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-0823bb0e-bb2f-4a3f-b4ef-291612d280b3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:12:51.475394 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0823bb0e-bb2f-4a3f-b4ef-291612d280b3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:12:51.477794 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0823bb0e-bb2f-4a3f-b4ef-291612d280b3 service nova] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:12:51.481998 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 830/1647] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 9 12:12:51 2022] POST /identity/v3/projects => generated 363 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:12:51.485899 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afe02ec-01de-4048-9120-d0d865fa9225 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:12:51.513115 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 818/1648] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:12:51 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:12:51.583680 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:12:51.640624 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afe02ec-01de-4048-9120-d0d865fa9225 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:12:51.657656 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c58c7dd-8e9f-47ed-9610-ef6f8df5518c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:12:51.658407 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c58c7dd-8e9f-47ed-9610-ef6f8df5518c admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:51.658799 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c58c7dd-8e9f-47ed-9610-ef6f8df5518c admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:51.659205 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c58c7dd-8e9f-47ed-9610-ef6f8df5518c admin admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:51.660062 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c58c7dd-8e9f-47ed-9610-ef6f8df5518c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:12:51.663170 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c58c7dd-8e9f-47ed-9610-ef6f8df5518c admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:12:51.713752 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 831/1649] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:12:51 2022] POST /identity/v3/users => generated 282 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:12:51.720082 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0823bb0e-bb2f-4a3f-b4ef-291612d280b3 service nova] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:12:51.742177 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0823bb0e-bb2f-4a3f-b4ef-291612d280b3 service nova] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:12:51.752765 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbc6f44-dbb0-4700-ac27-728a9de294b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:12:51.754003 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbc6f44-dbb0-4700-ac27-728a9de294b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:51.754003 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbc6f44-dbb0-4700-ac27-728a9de294b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:51.754003 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbc6f44-dbb0-4700-ac27-728a9de294b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:51.755861 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbc6f44-dbb0-4700-ac27-728a9de294b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:12:51.760075 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbc6f44-dbb0-4700-ac27-728a9de294b1 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:12:51.793291 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 819/1650] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:12:51 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:12:51.797270 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c58c7dd-8e9f-47ed-9610-ef6f8df5518c admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:12:51.811391 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c58c7dd-8e9f-47ed-9610-ef6f8df5518c admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:12:51.823296 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2f60f5-bffc-40d5-8028-494fa96d66d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:12:51.824409 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2f60f5-bffc-40d5-8028-494fa96d66d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:51.824409 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2f60f5-bffc-40d5-8028-494fa96d66d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:51.824570 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2f60f5-bffc-40d5-8028-494fa96d66d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:51.825143 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2f60f5-bffc-40d5-8028-494fa96d66d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:12:51.829858 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2f60f5-bffc-40d5-8028-494fa96d66d9 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:12:51.856792 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 832/1651] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:12:51 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 12:12:51.861411 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbc6f44-dbb0-4700-ac27-728a9de294b1 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:12:51.876519 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbc6f44-dbb0-4700-ac27-728a9de294b1 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:12:51.891681 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b278627d-bb1d-4118-b6a7-530a5eab1f15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:12:51.892589 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b278627d-bb1d-4118-b6a7-530a5eab1f15 admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:51.892870 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b278627d-bb1d-4118-b6a7-530a5eab1f15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:51.893383 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b278627d-bb1d-4118-b6a7-530a5eab1f15 admin admin] PATH_INFO: `/v3/projects/9aa17fb3ef5e4ae8a4ae045ec79d77da/users/1e5605d58dd0479c80eb19076bcf7569/roles/eecd2f160dd844f5906f3d345780c501` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:51.941494 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b278627d-bb1d-4118-b6a7-530a5eab1f15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9aa17fb3ef5e4ae8a4ae045ec79d77da, user_id=1e5605d58dd0479c80eb19076bcf7569, role_id=eecd2f160dd844f5906f3d345780c501)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:12:51.956998 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b278627d-bb1d-4118-b6a7-530a5eab1f15 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:12:52.003941 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 820/1652] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:12:51 2022] PUT /identity/v3/projects/9aa17fb3ef5e4ae8a4ae045ec79d77da/users/1e5605d58dd0479c80eb19076bcf7569/roles/eecd2f160dd844f5906f3d345780c501 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:12:52.017772 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259090f5-52b9-4064-b28f-ed33da975462 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:52.018088 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259090f5-52b9-4064-b28f-ed33da975462 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:52.018867 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259090f5-52b9-4064-b28f-ed33da975462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:52.127616 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-259090f5-52b9-4064-b28f-ed33da975462 None None] MFA Rules not processed for user `1e5605d58dd0479c80eb19076bcf7569`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:12:52.134828 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-259090f5-52b9-4064-b28f-ed33da975462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:12:52.389079 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 833/1653] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:12:52 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 374 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:12:52.394140 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b278627d-bb1d-4118-b6a7-530a5eab1f15 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:12:52.594075 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58571bdd-02a1-43d1-9ceb-9364ea2fee98 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:52.594367 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58571bdd-02a1-43d1-9ceb-9364ea2fee98 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:52.594651 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58571bdd-02a1-43d1-9ceb-9364ea2fee98 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:52.596127 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 834/1654] 10.222.0.49 () {54 vars in 912 bytes} [Fri Sep 9 12:12:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 12:12:52.605181 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ba01a7-7886-4e78-a579-8805b686fdc2 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:52.605324 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ba01a7-7886-4e78-a579-8805b686fdc2 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:52.605946 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ba01a7-7886-4e78-a579-8805b686fdc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:52.621192 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b278627d-bb1d-4118-b6a7-530a5eab1f15 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:12:52.649895 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb7cab7-dbf3-449d-9c14-697ce5b3a080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:12:52.650543 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb7cab7-dbf3-449d-9c14-697ce5b3a080 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:52.650875 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb7cab7-dbf3-449d-9c14-697ce5b3a080 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:52.651208 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb7cab7-dbf3-449d-9c14-697ce5b3a080 admin admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:52.656672 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb7cab7-dbf3-449d-9c14-697ce5b3a080 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:12:52.658836 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb7cab7-dbf3-449d-9c14-697ce5b3a080 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:12:52.721375 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-91ba01a7-7886-4e78-a579-8805b686fdc2 None None] MFA Rules not processed for user `992de6f86b42495dafe73cb2c393fb4e`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:12:52.722924 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-91ba01a7-7886-4e78-a579-8805b686fdc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:12:52.726703 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 821/1655] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 9 12:12:52 2022] POST /identity/v3/projects => generated 363 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:12:52.731663 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb7cab7-dbf3-449d-9c14-697ce5b3a080 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:12:52.868995 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb7cab7-dbf3-449d-9c14-697ce5b3a080 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:12:52.882364 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb67586-84bb-43a3-98f2-0ca1a36f8a0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:12:52.883132 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb67586-84bb-43a3-98f2-0ca1a36f8a0e admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:52.883320 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb67586-84bb-43a3-98f2-0ca1a36f8a0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:52.883550 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb67586-84bb-43a3-98f2-0ca1a36f8a0e admin admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:52.884196 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb67586-84bb-43a3-98f2-0ca1a36f8a0e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:12:52.887152 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb67586-84bb-43a3-98f2-0ca1a36f8a0e admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:12:52.891113 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 835/1656] 10.222.0.49 () {58 vars in 1009 bytes} [Fri Sep 9 12:12:52 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:12:52.905449 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ba01a7-7886-4e78-a579-8805b686fdc2 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:12:52.906989 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-91ba01a7-7886-4e78-a579-8805b686fdc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:12:52.927894 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ba01a7-7886-4e78-a579-8805b686fdc2 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:12:52.943010 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 822/1657] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:12:52 2022] POST /identity/v3/users => generated 288 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:12:52.948828 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb67586-84bb-43a3-98f2-0ca1a36f8a0e admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:12:52.976226 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb67586-84bb-43a3-98f2-0ca1a36f8a0e admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:12:52.992978 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe6b3a3-d08d-4cd0-aa15-2815f40b878d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:12:52.993608 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe6b3a3-d08d-4cd0-aa15-2815f40b878d admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:52.993898 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe6b3a3-d08d-4cd0-aa15-2815f40b878d admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:52.994151 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe6b3a3-d08d-4cd0-aa15-2815f40b878d admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:52.994794 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe6b3a3-d08d-4cd0-aa15-2815f40b878d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:12:52.996964 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe6b3a3-d08d-4cd0-aa15-2815f40b878d admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:12:53.017844 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:12:53.039873 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 823/1658] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:12:52 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 12:12:53.040157 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33a89bf-129b-40b0-a304-9b1c0b6df7f6 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:53.040594 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33a89bf-129b-40b0-a304-9b1c0b6df7f6 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:53.041001 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33a89bf-129b-40b0-a304-9b1c0b6df7f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:53.065119 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:12:53.092278 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd09780f-e059-4c27-9b14-c555bdbc00a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:12:53.092918 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd09780f-e059-4c27-9b14-c555bdbc00a7 service neutron] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:53.099445 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd09780f-e059-4c27-9b14-c555bdbc00a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:53.099734 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd09780f-e059-4c27-9b14-c555bdbc00a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:53.121927 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd09780f-e059-4c27-9b14-c555bdbc00a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:12:53.121927 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd09780f-e059-4c27-9b14-c555bdbc00a7 service neutron] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:12:53.148729 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 836/1659] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:12:52 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:12:53.149589 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd09780f-e059-4c27-9b14-c555bdbc00a7 service neutron] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:12:53.172236 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd09780f-e059-4c27-9b14-c555bdbc00a7 service neutron] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:12:53.183957 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd688dc6-7cdc-43a1-aaff-675bb928a26d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:12:53.184646 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd688dc6-7cdc-43a1-aaff-675bb928a26d admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:53.184926 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd688dc6-7cdc-43a1-aaff-675bb928a26d admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:53.185203 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd688dc6-7cdc-43a1-aaff-675bb928a26d admin admin] PATH_INFO: `/v3/projects/3f798123adf849b7915b68e477bbf546/users/7b4c68d82b6f450ea8c2fffeaf00da6c/roles/3f2ca33902a04b6dbc2ef7d06d7289d3` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:53.203979 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-f33a89bf-129b-40b0-a304-9b1c0b6df7f6 None None] MFA Rules not processed for user `2e125f66d9c84e9797099065351a07da`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:12:53.205740 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-f33a89bf-129b-40b0-a304-9b1c0b6df7f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:12:53.237748 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd688dc6-7cdc-43a1-aaff-675bb928a26d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f798123adf849b7915b68e477bbf546, user_id=7b4c68d82b6f450ea8c2fffeaf00da6c, role_id=3f2ca33902a04b6dbc2ef7d06d7289d3)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:12:53.241233 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd688dc6-7cdc-43a1-aaff-675bb928a26d admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:12:53.281406 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 837/1660] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:12:53 2022] PUT /identity/v3/projects/3f798123adf849b7915b68e477bbf546/users/7b4c68d82b6f450ea8c2fffeaf00da6c/roles/3f2ca33902a04b6dbc2ef7d06d7289d3 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:12:53.282434 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd688dc6-7cdc-43a1-aaff-675bb928a26d admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:12:53.425431 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 824/1661] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:12:53 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 386 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:12:53.479003 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd688dc6-7cdc-43a1-aaff-675bb928a26d admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:12:53.493282 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55a06c1-ecf9-45d9-9879-8889c1fc1163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:12:53.493926 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55a06c1-ecf9-45d9-9879-8889c1fc1163 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:53.494207 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55a06c1-ecf9-45d9-9879-8889c1fc1163 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:53.494485 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55a06c1-ecf9-45d9-9879-8889c1fc1163 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:53.495202 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55a06c1-ecf9-45d9-9879-8889c1fc1163 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:12:53.496835 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55a06c1-ecf9-45d9-9879-8889c1fc1163 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:12:53.528758 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33a89bf-129b-40b0-a304-9b1c0b6df7f6 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:12:53.536590 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 838/1662] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:12:53 2022] GET /identity/v3/roles => generated 1420 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:12:53.542893 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33a89bf-129b-40b0-a304-9b1c0b6df7f6 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:12:53.550663 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad542749-1628-4409-96d4-0e8385c16631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:12:53.552860 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad542749-1628-4409-96d4-0e8385c16631 admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:53.553519 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad542749-1628-4409-96d4-0e8385c16631 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:53.554657 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad542749-1628-4409-96d4-0e8385c16631 admin admin] PATH_INFO: `/v3/projects/3f798123adf849b7915b68e477bbf546/users/7b4c68d82b6f450ea8c2fffeaf00da6c/roles/3f2ca33902a04b6dbc2ef7d06d7289d3` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:53.567839 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad542749-1628-4409-96d4-0e8385c16631 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f798123adf849b7915b68e477bbf546, user_id=7b4c68d82b6f450ea8c2fffeaf00da6c, role_id=3f2ca33902a04b6dbc2ef7d06d7289d3)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:12:53.576904 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad542749-1628-4409-96d4-0e8385c16631 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:12:53.616652 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 825/1663] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:12:53 2022] PUT /identity/v3/projects/3f798123adf849b7915b68e477bbf546/users/7b4c68d82b6f450ea8c2fffeaf00da6c/roles/3f2ca33902a04b6dbc2ef7d06d7289d3 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:12:53.630282 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4855a0-99bf-4f0d-8787-73c820372647 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:53.630756 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4855a0-99bf-4f0d-8787-73c820372647 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:53.631258 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4855a0-99bf-4f0d-8787-73c820372647 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:53.720393 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad542749-1628-4409-96d4-0e8385c16631 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:12:53.723087 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-ad542749-1628-4409-96d4-0e8385c16631 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:12:53.770814 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-5c4855a0-99bf-4f0d-8787-73c820372647 None None] MFA Rules not processed for user `7b4c68d82b6f450ea8c2fffeaf00da6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:12:53.775342 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-5c4855a0-99bf-4f0d-8787-73c820372647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:12:53.917135 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad542749-1628-4409-96d4-0e8385c16631 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:12:53.925819 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72fc07c-cc9e-44ab-9087-0c6657da18eb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70bac169d5cd43e1926f310344ac11fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:12:53.927282 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72fc07c-cc9e-44ab-9087-0c6657da18eb service nova] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:53.927951 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72fc07c-cc9e-44ab-9087-0c6657da18eb service nova] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:53.928618 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72fc07c-cc9e-44ab-9087-0c6657da18eb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:53.971762 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 839/1664] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:12:53 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 340 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:12:53.980408 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4855a0-99bf-4f0d-8787-73c820372647 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:12:53.984678 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-5c4855a0-99bf-4f0d-8787-73c820372647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:12:54.076563 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72fc07c-cc9e-44ab-9087-0c6657da18eb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:12:54.080066 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72fc07c-cc9e-44ab-9087-0c6657da18eb service nova] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:12:54.108248 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:12:54.119296 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 826/1665] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:12:53 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:12:54.169484 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4855a0-99bf-4f0d-8787-73c820372647 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:12:54.179546 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1033d0-6c38-432f-b1c5-3d46e87c24c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535c258100df40b58a5ee44b2c21d621', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:12:54.180186 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1033d0-6c38-432f-b1c5-3d46e87c24c6 service cinder] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:54.180469 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1033d0-6c38-432f-b1c5-3d46e87c24c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:54.180736 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1033d0-6c38-432f-b1c5-3d46e87c24c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:54.214400 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1033d0-6c38-432f-b1c5-3d46e87c24c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:12:54.215898 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1033d0-6c38-432f-b1c5-3d46e87c24c6 service cinder] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:12:54.266571 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 840/1666] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:12:53 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:12:54.674400 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72fc07c-cc9e-44ab-9087-0c6657da18eb service nova] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:12:54.677304 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-e72fc07c-cc9e-44ab-9087-0c6657da18eb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:12:54.728928 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:12:54.804817 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1033d0-6c38-432f-b1c5-3d46e87c24c6 service cinder] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:12:54.806741 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-6b1033d0-6c38-432f-b1c5-3d46e87c24c6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:12:54.974858 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1033d0-6c38-432f-b1c5-3d46e87c24c6 service cinder] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:12:54.981652 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaee67e6-88aa-4c77-8558-9f6f4cf6bd10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:12:54.982454 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaee67e6-88aa-4c77-8558-9f6f4cf6bd10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:54.982819 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaee67e6-88aa-4c77-8558-9f6f4cf6bd10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:54.985751 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaee67e6-88aa-4c77-8558-9f6f4cf6bd10 admin admin] PATH_INFO: `/v3/users/d3e5d2cb984744b3acd4e9445fe7e248` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:54.991085 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaee67e6-88aa-4c77-8558-9f6f4cf6bd10 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3e5d2cb984744b3acd4e9445fe7e248)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:12:54.991085 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: WARNING py.warnings [None req-eaee67e6-88aa-4c77-8558-9f6f4cf6bd10 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 12:12:54.991085 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: warnings.warn(msg) Sep 09 12:12:54.991085 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: Sep 09 12:12:54.991085 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaee67e6-88aa-4c77-8558-9f6f4cf6bd10 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:12:55.006350 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72fc07c-cc9e-44ab-9087-0c6657da18eb service nova] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:12:55.023675 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aa22e3-88c1-49cc-b5d6-3e4cd017e81d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3adb2f0583c4ff782049dacdb3ee556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:12:55.024314 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aa22e3-88c1-49cc-b5d6-3e4cd017e81d service placement] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:55.024594 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aa22e3-88c1-49cc-b5d6-3e4cd017e81d service placement] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:55.024868 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aa22e3-88c1-49cc-b5d6-3e4cd017e81d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:55.027378 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-33aa22e3-88c1-49cc-b5d6-3e4cd017e81d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:12:55.055361 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33aa22e3-88c1-49cc-b5d6-3e4cd017e81d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:12:55.057051 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33aa22e3-88c1-49cc-b5d6-3e4cd017e81d service placement] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:12:55.083275 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 827/1667] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:12:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:12:55.147922 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-eaee67e6-88aa-4c77-8558-9f6f4cf6bd10 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3e5d2cb984744b3acd4e9445fe7e248'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:12:55.205283 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-eaee67e6-88aa-4c77-8558-9f6f4cf6bd10 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3e5d2cb984744b3acd4e9445fe7e248'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:12:55.298828 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a96695-291b-4b94-b75f-f746864b7d4d None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:55.299123 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a96695-291b-4b94-b75f-f746864b7d4d None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:55.299699 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a96695-291b-4b94-b75f-f746864b7d4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:55.321464 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-eaee67e6-88aa-4c77-8558-9f6f4cf6bd10 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3e5d2cb984744b3acd4e9445fe7e248'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:12:55.356371 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-eaee67e6-88aa-4c77-8558-9f6f4cf6bd10 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3e5d2cb984744b3acd4e9445fe7e248'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:12:55.365049 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-eaee67e6-88aa-4c77-8558-9f6f4cf6bd10 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3e5d2cb984744b3acd4e9445fe7e248'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:12:55.365168 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-53a96695-291b-4b94-b75f-f746864b7d4d None None] MFA Rules not processed for user `31411178e5c342b7bb14a21dec62f540`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:12:55.366588 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-53a96695-291b-4b94-b75f-f746864b7d4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:12:55.373364 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 841/1668] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 9 12:12:54 2022] DELETE /identity/v3/users/d3e5d2cb984744b3acd4e9445fe7e248 => generated 0 bytes in 569 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:12:55.382142 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae319a0f-fabf-45bc-9368-2fd8605dd363 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:55.382433 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae319a0f-fabf-45bc-9368-2fd8605dd363 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:55.382711 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae319a0f-fabf-45bc-9368-2fd8605dd363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:55.460693 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-ae319a0f-fabf-45bc-9368-2fd8605dd363 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:12:55.464718 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-ae319a0f-fabf-45bc-9368-2fd8605dd363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:12:55.654424 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 828/1669] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:12:55 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 357 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:12:55.664567 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 842/1670] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:12:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:12:55.888286 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a96695-291b-4b94-b75f-f746864b7d4d None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:12:55.890486 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-53a96695-291b-4b94-b75f-f746864b7d4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:12:56.091529 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae319a0f-fabf-45bc-9368-2fd8605dd363 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:12:56.093266 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-ae319a0f-fabf-45bc-9368-2fd8605dd363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:12:56.112182 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:12:56.118479 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae319a0f-fabf-45bc-9368-2fd8605dd363 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:12:56.137443 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e87ec19-e3c1-447f-8091-3f1e862835f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:12:56.138794 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e87ec19-e3c1-447f-8091-3f1e862835f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:56.139392 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e87ec19-e3c1-447f-8091-3f1e862835f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:56.141080 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e87ec19-e3c1-447f-8091-3f1e862835f7 admin admin] PATH_INFO: `/v3/users/1e5605d58dd0479c80eb19076bcf7569` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:56.143793 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e87ec19-e3c1-447f-8091-3f1e862835f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e5605d58dd0479c80eb19076bcf7569)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:12:56.148229 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e87ec19-e3c1-447f-8091-3f1e862835f7 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:12:56.169024 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a96695-291b-4b94-b75f-f746864b7d4d None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:12:56.180380 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0210bbe-0eca-4dc1-9b8a-ddf0054b43af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:12:56.184189 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0210bbe-0eca-4dc1-9b8a-ddf0054b43af service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:56.184696 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0210bbe-0eca-4dc1-9b8a-ddf0054b43af service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:56.185113 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0210bbe-0eca-4dc1-9b8a-ddf0054b43af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:56.197056 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0210bbe-0eca-4dc1-9b8a-ddf0054b43af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:12:56.199729 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0210bbe-0eca-4dc1-9b8a-ddf0054b43af service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:12:56.216292 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 829/1671] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:12:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:12:56.217988 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43d9411-f0e2-4fcc-97a4-474a89e31b1d None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:56.218532 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43d9411-f0e2-4fcc-97a4-474a89e31b1d None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:56.219178 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43d9411-f0e2-4fcc-97a4-474a89e31b1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:56.269797 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-d43d9411-f0e2-4fcc-97a4-474a89e31b1d None None] MFA Rules not processed for user `ab948442f326433c8e9f5f90314627aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:12:56.271135 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-d43d9411-f0e2-4fcc-97a4-474a89e31b1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:12:56.314052 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-5e87ec19-e3c1-447f-8091-3f1e862835f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e5605d58dd0479c80eb19076bcf7569'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:12:56.362223 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-5e87ec19-e3c1-447f-8091-3f1e862835f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e5605d58dd0479c80eb19076bcf7569'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:12:56.385612 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:12:56.397146 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-5e87ec19-e3c1-447f-8091-3f1e862835f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e5605d58dd0479c80eb19076bcf7569'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:12:56.424574 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-5e87ec19-e3c1-447f-8091-3f1e862835f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e5605d58dd0479c80eb19076bcf7569'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:12:56.436568 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-5e87ec19-e3c1-447f-8091-3f1e862835f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e5605d58dd0479c80eb19076bcf7569'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:12:56.455430 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 843/1672] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 9 12:12:56 2022] DELETE /identity/v3/users/1e5605d58dd0479c80eb19076bcf7569 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:12:56.468959 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b66db5-2694-474d-9828-83a612e46528 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:56.468959 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b66db5-2694-474d-9828-83a612e46528 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:56.468959 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b66db5-2694-474d-9828-83a612e46528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:56.469469 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 830/1673] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:12:56 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:12:56.471635 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d45827-709a-4c6d-bc4c-1ab19560c0e3 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:56.471917 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d45827-709a-4c6d-bc4c-1ab19560c0e3 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:56.472186 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d45827-709a-4c6d-bc4c-1ab19560c0e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:56.522869 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-81d45827-709a-4c6d-bc4c-1ab19560c0e3 None None] MFA Rules not processed for user `ab948442f326433c8e9f5f90314627aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:12:56.525464 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-81d45827-709a-4c6d-bc4c-1ab19560c0e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:12:56.527937 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-c2b66db5-2694-474d-9828-83a612e46528 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:12:56.530627 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-c2b66db5-2694-474d-9828-83a612e46528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:12:56.744556 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 831/1674] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:12:56 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:12:56.752331 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d45827-709a-4c6d-bc4c-1ab19560c0e3 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:12:56.753902 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-81d45827-709a-4c6d-bc4c-1ab19560c0e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:12:56.761475 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 844/1675] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:12:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:12:56.773139 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b66db5-2694-474d-9828-83a612e46528 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:12:56.774780 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-c2b66db5-2694-474d-9828-83a612e46528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:12:56.904328 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b66db5-2694-474d-9828-83a612e46528 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:12:56.913835 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05772769-1c34-46e8-a47b-9edcf67ae08a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:12:56.916421 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05772769-1c34-46e8-a47b-9edcf67ae08a service neutron] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:56.917897 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05772769-1c34-46e8-a47b-9edcf67ae08a service neutron] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:56.918463 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05772769-1c34-46e8-a47b-9edcf67ae08a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:56.937919 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05772769-1c34-46e8-a47b-9edcf67ae08a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:12:56.939770 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05772769-1c34-46e8-a47b-9edcf67ae08a service neutron] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:12:56.947677 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d45827-709a-4c6d-bc4c-1ab19560c0e3 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:12:56.957539 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 845/1676] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:12:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:12:56.976188 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9757524-f5dc-4817-861f-97c0af84886d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70bac169d5cd43e1926f310344ac11fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:12:56.978100 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9757524-f5dc-4817-861f-97c0af84886d service nova] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:56.978401 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9757524-f5dc-4817-861f-97c0af84886d service nova] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:56.978636 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9757524-f5dc-4817-861f-97c0af84886d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:56.984249 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-f9757524-f5dc-4817-861f-97c0af84886d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:12:57.022573 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9757524-f5dc-4817-861f-97c0af84886d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:12:57.024300 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9757524-f5dc-4817-861f-97c0af84886d service nova] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:12:57.042697 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 832/1677] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:12:56 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:12:57.188781 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4f3f52-d669-46a6-8116-b55c4841abbe None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:57.189270 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4f3f52-d669-46a6-8116-b55c4841abbe None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:57.189388 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4f3f52-d669-46a6-8116-b55c4841abbe None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:57.190684 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 846/1678] 10.222.0.49 () {54 vars in 912 bytes} [Fri Sep 9 12:12:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:12:57.205430 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769334cf-68a8-4865-99d2-cfda89626ac3 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:57.206014 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769334cf-68a8-4865-99d2-cfda89626ac3 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:57.206438 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769334cf-68a8-4865-99d2-cfda89626ac3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:57.307973 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-769334cf-68a8-4865-99d2-cfda89626ac3 None None] MFA Rules not processed for user `992de6f86b42495dafe73cb2c393fb4e`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:12:57.314532 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-769334cf-68a8-4865-99d2-cfda89626ac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:12:57.347554 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 833/1679] 10.222.0.49 () {58 vars in 1009 bytes} [Fri Sep 9 12:12:57 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:12:57.650559 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4f3f52-d669-46a6-8116-b55c4841abbe None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:12:57.654806 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-0d4f3f52-d669-46a6-8116-b55c4841abbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:12:57.711304 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4f3f52-d669-46a6-8116-b55c4841abbe None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:12:57.735529 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de7b917-d8d7-4909-8843-7e4a45fa9491 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:12:57.741450 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de7b917-d8d7-4909-8843-7e4a45fa9491 service neutron] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:57.755669 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de7b917-d8d7-4909-8843-7e4a45fa9491 service neutron] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:57.756166 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de7b917-d8d7-4909-8843-7e4a45fa9491 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:57.808111 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de7b917-d8d7-4909-8843-7e4a45fa9491 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:12:57.810124 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de7b917-d8d7-4909-8843-7e4a45fa9491 service neutron] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:12:57.826221 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769334cf-68a8-4865-99d2-cfda89626ac3 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:12:57.828212 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-769334cf-68a8-4865-99d2-cfda89626ac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:12:57.858464 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 847/1680] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:12:57 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:12:57.884595 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769334cf-68a8-4865-99d2-cfda89626ac3 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:12:57.902576 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacb4817-525e-427a-8f08-9df59345ee82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:12:57.903639 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacb4817-525e-427a-8f08-9df59345ee82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:57.905289 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacb4817-525e-427a-8f08-9df59345ee82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:57.905594 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacb4817-525e-427a-8f08-9df59345ee82 admin admin] PATH_INFO: `/v3/projects/5cc0bd37a3024948b849e396014778e9` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:57.908971 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacb4817-525e-427a-8f08-9df59345ee82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5cc0bd37a3024948b849e396014778e9)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:12:57.911956 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: WARNING py.warnings [None req-eacb4817-525e-427a-8f08-9df59345ee82 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 12:12:57.911956 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: warnings.warn(msg) Sep 09 12:12:57.911956 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: Sep 09 12:12:57.915290 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacb4817-525e-427a-8f08-9df59345ee82 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:12:58.109158 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-eacb4817-525e-427a-8f08-9df59345ee82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5cc0bd37a3024948b849e396014778e9', 'request_id': 'req-eacb4817-525e-427a-8f08-9df59345ee82'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:12:58.128473 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-eacb4817-525e-427a-8f08-9df59345ee82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5cc0bd37a3024948b849e396014778e9', 'request_id': 'req-eacb4817-525e-427a-8f08-9df59345ee82'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:12:58.140350 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:12:58.157158 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-eacb4817-525e-427a-8f08-9df59345ee82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5cc0bd37a3024948b849e396014778e9', 'request_id': 'req-eacb4817-525e-427a-8f08-9df59345ee82'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:12:58.255897 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-eacb4817-525e-427a-8f08-9df59345ee82 admin admin] The token cache is being invalidate because project 5cc0bd37a3024948b849e396014778e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:12:58.256818 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-eacb4817-525e-427a-8f08-9df59345ee82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:12:58.258781 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-eacb4817-525e-427a-8f08-9df59345ee82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:12:58.262314 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 834/1681] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 9 12:12:57 2022] DELETE /identity/v3/projects/5cc0bd37a3024948b849e396014778e9 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:12:58.267232 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe5606c-b18d-4b18-8701-9ac855d5cb9a None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:58.269565 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe5606c-b18d-4b18-8701-9ac855d5cb9a None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:58.270138 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe5606c-b18d-4b18-8701-9ac855d5cb9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:58.308145 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:12:58.324575 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-abe5606c-b18d-4b18-8701-9ac855d5cb9a None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:12:58.328414 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-abe5606c-b18d-4b18-8701-9ac855d5cb9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:12:58.521355 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 848/1682] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:12:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:12:58.530031 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bac38ac-cf92-43f4-a94b-a2d6a6329f3e None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:58.530638 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bac38ac-cf92-43f4-a94b-a2d6a6329f3e None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:58.531134 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bac38ac-cf92-43f4-a94b-a2d6a6329f3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:58.576525 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-5bac38ac-cf92-43f4-a94b-a2d6a6329f3e None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:12:58.577945 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-5bac38ac-cf92-43f4-a94b-a2d6a6329f3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:12:58.625484 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 835/1683] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:12:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:12:58.639462 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe5606c-b18d-4b18-8701-9ac855d5cb9a None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:12:58.659364 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe5606c-b18d-4b18-8701-9ac855d5cb9a None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:12:58.679730 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5049a0b-7759-48fc-ab79-30bdcfbb27e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:12:58.680384 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5049a0b-7759-48fc-ab79-30bdcfbb27e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:58.680796 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5049a0b-7759-48fc-ab79-30bdcfbb27e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:58.681118 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5049a0b-7759-48fc-ab79-30bdcfbb27e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:58.681799 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5049a0b-7759-48fc-ab79-30bdcfbb27e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:12:58.682099 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5049a0b-7759-48fc-ab79-30bdcfbb27e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:12:58.684846 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5049a0b-7759-48fc-ab79-30bdcfbb27e0 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:12:58.727648 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 849/1684] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 9 12:12:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:12:58.732734 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bac38ac-cf92-43f4-a94b-a2d6a6329f3e None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:12:58.750224 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bac38ac-cf92-43f4-a94b-a2d6a6329f3e None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:12:58.812815 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577bae3e-b800-4cf3-be85-9b2933139f9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:12:58.813904 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577bae3e-b800-4cf3-be85-9b2933139f9f admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:58.814185 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577bae3e-b800-4cf3-be85-9b2933139f9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:58.814451 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577bae3e-b800-4cf3-be85-9b2933139f9f admin admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:58.815462 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577bae3e-b800-4cf3-be85-9b2933139f9f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:12:58.818810 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577bae3e-b800-4cf3-be85-9b2933139f9f admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:12:58.922468 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 836/1685] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 9 12:12:58 2022] POST /identity/v3/projects => generated 403 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:12:58.926465 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5049a0b-7759-48fc-ab79-30bdcfbb27e0 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:12:58.946051 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5049a0b-7759-48fc-ab79-30bdcfbb27e0 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:12:59.174195 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3010defe-325d-4090-987f-6353daee2d6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:12:59.177507 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3010defe-325d-4090-987f-6353daee2d6a admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:59.177820 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3010defe-325d-4090-987f-6353daee2d6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:59.179210 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3010defe-325d-4090-987f-6353daee2d6a admin admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:59.183007 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3010defe-325d-4090-987f-6353daee2d6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:12:59.191041 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3010defe-325d-4090-987f-6353daee2d6a admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:12:59.268574 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 850/1686] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:12:58 2022] POST /identity/v3/users => generated 302 bytes in 343 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:12:59.274504 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577bae3e-b800-4cf3-be85-9b2933139f9f admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:12:59.290817 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577bae3e-b800-4cf3-be85-9b2933139f9f admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:12:59.297553 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6dd0ae-c218-4821-ace3-3843f3726fbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:12:59.299088 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6dd0ae-c218-4821-ace3-3843f3726fbd admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:59.301145 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6dd0ae-c218-4821-ace3-3843f3726fbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:59.301632 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6dd0ae-c218-4821-ace3-3843f3726fbd admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:59.302685 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6dd0ae-c218-4821-ace3-3843f3726fbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:12:59.305109 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6dd0ae-c218-4821-ace3-3843f3726fbd admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:12:59.321403 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 837/1687] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:12:59 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:12:59.326828 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3010defe-325d-4090-987f-6353daee2d6a admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:12:59.341697 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3010defe-325d-4090-987f-6353daee2d6a admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:12:59.348888 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8427815-5257-4337-ac6f-f335224835ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:12:59.350221 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8427815-5257-4337-ac6f-f335224835ab admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:59.350683 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8427815-5257-4337-ac6f-f335224835ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:59.351713 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8427815-5257-4337-ac6f-f335224835ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:59.352679 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8427815-5257-4337-ac6f-f335224835ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:12:59.355527 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8427815-5257-4337-ac6f-f335224835ab admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:12:59.370932 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 851/1688] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:12:59 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:12:59.375720 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6dd0ae-c218-4821-ace3-3843f3726fbd admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:12:59.425021 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8427815-5257-4337-ac6f-f335224835ab admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:12:59.432128 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-e8427815-5257-4337-ac6f-f335224835ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:12:59.432994 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6dd0ae-c218-4821-ace3-3843f3726fbd admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:12:59.467008 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33320e84-b710-40cb-a2ed-43b721762478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:12:59.484005 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33320e84-b710-40cb-a2ed-43b721762478 admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:59.486464 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33320e84-b710-40cb-a2ed-43b721762478 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:59.486464 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33320e84-b710-40cb-a2ed-43b721762478 admin admin] PATH_INFO: `/v3/projects/73d7d7afa06b4426970d12058524f938/users/3360a42d91bb4360b9ec0b02a4a91629/roles/eecd2f160dd844f5906f3d345780c501` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:59.515567 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8427815-5257-4337-ac6f-f335224835ab admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:12:59.534322 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb33f25f-c886-40c1-9a91-34478e9e66ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:12:59.536638 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb33f25f-c886-40c1-9a91-34478e9e66ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:59.538977 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb33f25f-c886-40c1-9a91-34478e9e66ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:59.539353 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb33f25f-c886-40c1-9a91-34478e9e66ea admin admin] PATH_INFO: `/v3/projects/9aa17fb3ef5e4ae8a4ae045ec79d77da` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:59.544801 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb33f25f-c886-40c1-9a91-34478e9e66ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=9aa17fb3ef5e4ae8a4ae045ec79d77da)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:12:59.547256 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb33f25f-c886-40c1-9a91-34478e9e66ea admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:12:59.554596 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33320e84-b710-40cb-a2ed-43b721762478 admin admin] RBAC: Authorizing `identity:create_grant(project_id=73d7d7afa06b4426970d12058524f938, user_id=3360a42d91bb4360b9ec0b02a4a91629, role_id=eecd2f160dd844f5906f3d345780c501)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:12:59.562631 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33320e84-b710-40cb-a2ed-43b721762478 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:12:59.602184 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 838/1689] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:12:59 2022] PUT /identity/v3/projects/73d7d7afa06b4426970d12058524f938/users/3360a42d91bb4360b9ec0b02a4a91629/roles/eecd2f160dd844f5906f3d345780c501 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:12:59.602184 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d1654f-a3e5-474d-aed1-280fa2c6d0f0 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:59.602184 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d1654f-a3e5-474d-aed1-280fa2c6d0f0 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:59.602184 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d1654f-a3e5-474d-aed1-280fa2c6d0f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:59.693064 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-bb33f25f-c886-40c1-9a91-34478e9e66ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9aa17fb3ef5e4ae8a4ae045ec79d77da', 'request_id': 'req-bb33f25f-c886-40c1-9a91-34478e9e66ea'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:12:59.708024 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-bb33f25f-c886-40c1-9a91-34478e9e66ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9aa17fb3ef5e4ae8a4ae045ec79d77da', 'request_id': 'req-bb33f25f-c886-40c1-9a91-34478e9e66ea'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:12:59.727125 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-bb33f25f-c886-40c1-9a91-34478e9e66ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9aa17fb3ef5e4ae8a4ae045ec79d77da', 'request_id': 'req-bb33f25f-c886-40c1-9a91-34478e9e66ea'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:12:59.740782 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:12:59.761165 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-44d1654f-a3e5-474d-aed1-280fa2c6d0f0 None None] MFA Rules not processed for user `e368b4e668c54e25bff80a8e8a8c3588`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:12:59.762553 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-44d1654f-a3e5-474d-aed1-280fa2c6d0f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:12:59.763482 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-bb33f25f-c886-40c1-9a91-34478e9e66ea admin admin] The token cache is being invalidate because project 9aa17fb3ef5e4ae8a4ae045ec79d77da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81591) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:12:59.763989 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-bb33f25f-c886-40c1-9a91-34478e9e66ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:12:59.765249 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-bb33f25f-c886-40c1-9a91-34478e9e66ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:12:59.769270 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 852/1690] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 9 12:12:59 2022] DELETE /identity/v3/projects/9aa17fb3ef5e4ae8a4ae045ec79d77da => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:12:59.774374 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3f7d05-519e-472e-acfb-3e8fde5a24a5 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:12:59.775521 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3f7d05-519e-472e-acfb-3e8fde5a24a5 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:12:59.775521 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3f7d05-519e-472e-acfb-3e8fde5a24a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:12:59.872760 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-1b3f7d05-519e-472e-acfb-3e8fde5a24a5 None None] MFA Rules not processed for user `3360a42d91bb4360b9ec0b02a4a91629`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:12:59.876391 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-1b3f7d05-519e-472e-acfb-3e8fde5a24a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:12:59.967020 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 839/1691] 10.222.0.49 () {58 vars in 1009 bytes} [Fri Sep 9 12:12:59 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 374 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:12:59.971250 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d1654f-a3e5-474d-aed1-280fa2c6d0f0 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:12:59.974586 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-44d1654f-a3e5-474d-aed1-280fa2c6d0f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:00.075616 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 853/1692] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:12:59 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:13:00.078249 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3f7d05-519e-472e-acfb-3e8fde5a24a5 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:00.079918 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-1b3f7d05-519e-472e-acfb-3e8fde5a24a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:00.137967 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:INSERT=3,keystone:DELETE=1,keystone:SELECT=96 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:00.158442 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d1654f-a3e5-474d-aed1-280fa2c6d0f0 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:00.175670 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6369227-05d7-42ae-85d7-eeb9fc0cd491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:00.176335 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6369227-05d7-42ae-85d7-eeb9fc0cd491 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:00.176615 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6369227-05d7-42ae-85d7-eeb9fc0cd491 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:00.176889 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6369227-05d7-42ae-85d7-eeb9fc0cd491 admin admin] PATH_INFO: `/v3/users/7b4c68d82b6f450ea8c2fffeaf00da6c` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:00.184843 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6369227-05d7-42ae-85d7-eeb9fc0cd491 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b4c68d82b6f450ea8c2fffeaf00da6c)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:00.187145 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6369227-05d7-42ae-85d7-eeb9fc0cd491 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:00.252020 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3f7d05-519e-472e-acfb-3e8fde5a24a5 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:00.257380 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adaca49-1e7e-4b40-923f-29ff6d1d9479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:00.258154 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adaca49-1e7e-4b40-923f-29ff6d1d9479 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:00.258494 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adaca49-1e7e-4b40-923f-29ff6d1d9479 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:00.261019 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adaca49-1e7e-4b40-923f-29ff6d1d9479 admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:00.261019 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adaca49-1e7e-4b40-923f-29ff6d1d9479 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:00.263282 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adaca49-1e7e-4b40-923f-29ff6d1d9479 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:00.340940 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 854/1693] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 9 12:13:00 2022] POST /identity/v3/projects => generated 405 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:13:00.340940 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adaca49-1e7e-4b40-923f-29ff6d1d9479 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:00.343752 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-1adaca49-1e7e-4b40-923f-29ff6d1d9479 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:00.351174 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-e6369227-05d7-42ae-85d7-eeb9fc0cd491 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b4c68d82b6f450ea8c2fffeaf00da6c'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:00.386532 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-e6369227-05d7-42ae-85d7-eeb9fc0cd491 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b4c68d82b6f450ea8c2fffeaf00da6c'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:00.441929 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-e6369227-05d7-42ae-85d7-eeb9fc0cd491 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b4c68d82b6f450ea8c2fffeaf00da6c'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:00.460952 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-e6369227-05d7-42ae-85d7-eeb9fc0cd491 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b4c68d82b6f450ea8c2fffeaf00da6c'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:00.461861 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-e6369227-05d7-42ae-85d7-eeb9fc0cd491 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b4c68d82b6f450ea8c2fffeaf00da6c'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:00.472016 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 840/1694] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 9 12:12:59 2022] DELETE /identity/v3/users/7b4c68d82b6f450ea8c2fffeaf00da6c => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:13:00.473235 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6369227-05d7-42ae-85d7-eeb9fc0cd491 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:00.480657 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-e6369227-05d7-42ae-85d7-eeb9fc0cd491 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:00.533193 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adaca49-1e7e-4b40-923f-29ff6d1d9479 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:00.612849 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:00.811047 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6369227-05d7-42ae-85d7-eeb9fc0cd491 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:00.829377 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad4cff2-3c77-4096-8721-29be3bc08a16 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e652ba827849b5a3fc1213d04a1b76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:00.830257 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad4cff2-3c77-4096-8721-29be3bc08a16 service swift] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:00.830708 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad4cff2-3c77-4096-8721-29be3bc08a16 service swift] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:00.831149 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad4cff2-3c77-4096-8721-29be3bc08a16 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:00.833238 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-0ad4cff2-3c77-4096-8721-29be3bc08a16 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:00.859539 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a430e69c-d900-423d-8709-8bf10f9b394e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:00.860212 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a430e69c-d900-423d-8709-8bf10f9b394e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:00.860489 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a430e69c-d900-423d-8709-8bf10f9b394e admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:00.860846 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a430e69c-d900-423d-8709-8bf10f9b394e admin admin] PATH_INFO: `/v3/users/c18cf3d809c24930a4b08930a2835a7f` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:00.867894 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a430e69c-d900-423d-8709-8bf10f9b394e admin admin] RBAC: Authorizing `identity:delete_user(user_id=c18cf3d809c24930a4b08930a2835a7f)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:00.870945 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a430e69c-d900-423d-8709-8bf10f9b394e admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:00.979750 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:00.984631 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad4cff2-3c77-4096-8721-29be3bc08a16 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:00.986254 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad4cff2-3c77-4096-8721-29be3bc08a16 service swift] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:00.993623 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-a430e69c-d900-423d-8709-8bf10f9b394e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c18cf3d809c24930a4b08930a2835a7f'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:01.018370 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-a430e69c-d900-423d-8709-8bf10f9b394e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c18cf3d809c24930a4b08930a2835a7f'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:01.021615 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 841/1695] 10.222.0.49 () {60 vars in 1384 bytes} [Fri Sep 9 12:13:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 549 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 12:13:01.022680 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad4cff2-3c77-4096-8721-29be3bc08a16 service swift] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:01.029295 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-0ad4cff2-3c77-4096-8721-29be3bc08a16 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:01.079950 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad4cff2-3c77-4096-8721-29be3bc08a16 service swift] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:01.082667 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-a430e69c-d900-423d-8709-8bf10f9b394e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c18cf3d809c24930a4b08930a2835a7f'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:01.096010 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-a430e69c-d900-423d-8709-8bf10f9b394e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c18cf3d809c24930a4b08930a2835a7f'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:01.097482 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-a430e69c-d900-423d-8709-8bf10f9b394e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c18cf3d809c24930a4b08930a2835a7f'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:01.100872 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c769f48b-f6bb-4a1f-85b7-86763bc58096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:01.104637 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c769f48b-f6bb-4a1f-85b7-86763bc58096 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:01.105579 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c769f48b-f6bb-4a1f-85b7-86763bc58096 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:01.107427 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c769f48b-f6bb-4a1f-85b7-86763bc58096 admin admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:01.108726 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c769f48b-f6bb-4a1f-85b7-86763bc58096 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:01.111287 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c769f48b-f6bb-4a1f-85b7-86763bc58096 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:01.114377 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 855/1696] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 9 12:13:00 2022] DELETE /identity/v3/users/c18cf3d809c24930a4b08930a2835a7f => generated 0 bytes in 776 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:13:01.120835 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5946154b-d0d0-45d9-a12e-2b68dfe1c785 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:01.121233 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5946154b-d0d0-45d9-a12e-2b68dfe1c785 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:01.121688 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5946154b-d0d0-45d9-a12e-2b68dfe1c785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:01.200612 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 842/1697] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:13:01 2022] POST /identity/v3/users => generated 309 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:13:01.201821 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c769f48b-f6bb-4a1f-85b7-86763bc58096 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:01.208866 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-c769f48b-f6bb-4a1f-85b7-86763bc58096 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:01.223802 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-5946154b-d0d0-45d9-a12e-2b68dfe1c785 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:13:01.225146 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-5946154b-d0d0-45d9-a12e-2b68dfe1c785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:01.277552 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c769f48b-f6bb-4a1f-85b7-86763bc58096 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:01.423436 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 856/1698] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:13:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:13:01.485561 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5946154b-d0d0-45d9-a12e-2b68dfe1c785 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:01.491191 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-5946154b-d0d0-45d9-a12e-2b68dfe1c785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:01.580762 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6903c9-845a-46a2-920c-91a699f95573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:01.581388 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6903c9-845a-46a2-920c-91a699f95573 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:01.587536 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6903c9-845a-46a2-920c-91a699f95573 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:01.587823 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6903c9-845a-46a2-920c-91a699f95573 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:01.593157 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6903c9-845a-46a2-920c-91a699f95573 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:01.600571 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6903c9-845a-46a2-920c-91a699f95573 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:01.618162 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 843/1699] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:13:01 2022] GET /identity/v3/roles => generated 1420 bytes in 417 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:13:01.625435 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6903c9-845a-46a2-920c-91a699f95573 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:01.633253 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:01.645950 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6903c9-845a-46a2-920c-91a699f95573 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:01.671584 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fd2389-dcc1-4f21-99b4-bbd0b4caef89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:01.676677 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fd2389-dcc1-4f21-99b4-bbd0b4caef89 admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:01.677403 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fd2389-dcc1-4f21-99b4-bbd0b4caef89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:01.682771 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fd2389-dcc1-4f21-99b4-bbd0b4caef89 admin admin] PATH_INFO: `/v3/projects/4e5b3edbfe2743a0854e53923d366cf2/users/2e7306305f884561bc053ab70e63570f/roles/3f2ca33902a04b6dbc2ef7d06d7289d3` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:01.708997 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5946154b-d0d0-45d9-a12e-2b68dfe1c785 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:01.736618 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd7d872-f4fc-4221-87f0-b2cffc24a262 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:01.740099 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd7d872-f4fc-4221-87f0-b2cffc24a262 service neutron] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:01.740099 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd7d872-f4fc-4221-87f0-b2cffc24a262 service neutron] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:01.741819 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd7d872-f4fc-4221-87f0-b2cffc24a262 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:01.770990 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fd2389-dcc1-4f21-99b4-bbd0b4caef89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e5b3edbfe2743a0854e53923d366cf2, user_id=2e7306305f884561bc053ab70e63570f, role_id=3f2ca33902a04b6dbc2ef7d06d7289d3)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:01.783738 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd7d872-f4fc-4221-87f0-b2cffc24a262 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:01.784191 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fd2389-dcc1-4f21-99b4-bbd0b4caef89 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:01.790460 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd7d872-f4fc-4221-87f0-b2cffc24a262 service neutron] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:01.816414 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 844/1700] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:13:01 2022] PUT /identity/v3/projects/4e5b3edbfe2743a0854e53923d366cf2/users/2e7306305f884561bc053ab70e63570f/roles/3f2ca33902a04b6dbc2ef7d06d7289d3 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:13:01.819917 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 857/1701] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:13:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:13:01.821303 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fd2389-dcc1-4f21-99b4-bbd0b4caef89 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:01.908188 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f8c177-ceed-4360-9c7b-299f7139299b None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:01.908683 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f8c177-ceed-4360-9c7b-299f7139299b None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:01.909151 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f8c177-ceed-4360-9c7b-299f7139299b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:01.977852 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-52f8c177-ceed-4360-9c7b-299f7139299b None None] MFA Rules not processed for user `2e125f66d9c84e9797099065351a07da`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:13:01.983985 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-52f8c177-ceed-4360-9c7b-299f7139299b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:02.034996 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fd2389-dcc1-4f21-99b4-bbd0b4caef89 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:02.047261 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4a3b29-466b-4fa0-bd65-a366ed8f7857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:02.047696 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4a3b29-466b-4fa0-bd65-a366ed8f7857 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:02.049048 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4a3b29-466b-4fa0-bd65-a366ed8f7857 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:02.049603 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4a3b29-466b-4fa0-bd65-a366ed8f7857 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:02.051024 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4a3b29-466b-4fa0-bd65-a366ed8f7857 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:02.055506 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4a3b29-466b-4fa0-bd65-a366ed8f7857 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:02.094753 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 845/1702] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:13:01 2022] GET /identity/v3/roles => generated 1420 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 12:13:02.099085 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4a3b29-466b-4fa0-bd65-a366ed8f7857 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:02.115584 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4a3b29-466b-4fa0-bd65-a366ed8f7857 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:02.123665 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628ddf77-9387-4450-a3cb-ab400a2ab05d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:02.124612 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628ddf77-9387-4450-a3cb-ab400a2ab05d admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:02.125045 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628ddf77-9387-4450-a3cb-ab400a2ab05d admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:02.125405 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628ddf77-9387-4450-a3cb-ab400a2ab05d admin admin] PATH_INFO: `/v3/projects/4e5b3edbfe2743a0854e53923d366cf2/users/2e7306305f884561bc053ab70e63570f/roles/3f2ca33902a04b6dbc2ef7d06d7289d3` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:02.128241 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628ddf77-9387-4450-a3cb-ab400a2ab05d admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e5b3edbfe2743a0854e53923d366cf2, user_id=2e7306305f884561bc053ab70e63570f, role_id=3f2ca33902a04b6dbc2ef7d06d7289d3)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:02.135092 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628ddf77-9387-4450-a3cb-ab400a2ab05d admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:02.156476 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 846/1703] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:13:02 2022] PUT /identity/v3/projects/4e5b3edbfe2743a0854e53923d366cf2/users/2e7306305f884561bc053ab70e63570f/roles/3f2ca33902a04b6dbc2ef7d06d7289d3 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:13:02.169383 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f742ef3e-d471-4dd8-9886-ed556faa8f10 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:02.169892 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f742ef3e-d471-4dd8-9886-ed556faa8f10 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:02.170231 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f742ef3e-d471-4dd8-9886-ed556faa8f10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:02.198703 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 858/1704] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:13:01 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:13:02.215977 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-f742ef3e-d471-4dd8-9886-ed556faa8f10 None None] MFA Rules not processed for user `2e7306305f884561bc053ab70e63570f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:13:02.218101 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-f742ef3e-d471-4dd8-9886-ed556faa8f10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:02.301416 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:02.495544 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f8c177-ceed-4360-9c7b-299f7139299b None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:02.495544 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-52f8c177-ceed-4360-9c7b-299f7139299b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:02.507203 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 847/1705] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:13:02 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:13:02.836632 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f8c177-ceed-4360-9c7b-299f7139299b None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:02.845604 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f742ef3e-d471-4dd8-9886-ed556faa8f10 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:02.848393 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-f742ef3e-d471-4dd8-9886-ed556faa8f10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:03.100777 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48be25b2-5159-474d-9a4b-0a7b26b950c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535c258100df40b58a5ee44b2c21d621', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:03.100777 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48be25b2-5159-474d-9a4b-0a7b26b950c0 service cinder] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:03.100777 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48be25b2-5159-474d-9a4b-0a7b26b950c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:03.100777 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48be25b2-5159-474d-9a4b-0a7b26b950c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:03.115861 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48be25b2-5159-474d-9a4b-0a7b26b950c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:03.130432 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48be25b2-5159-474d-9a4b-0a7b26b950c0 service cinder] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:03.141493 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f742ef3e-d471-4dd8-9886-ed556faa8f10 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:03.157821 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 859/1706] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:13:02 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 665 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:13:03.165887 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf968cd-d49b-483d-b0ff-f8b2d4721126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:03.174321 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf968cd-d49b-483d-b0ff-f8b2d4721126 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:03.174595 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf968cd-d49b-483d-b0ff-f8b2d4721126 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:03.174872 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf968cd-d49b-483d-b0ff-f8b2d4721126 admin admin] PATH_INFO: `/v3/projects/3f798123adf849b7915b68e477bbf546` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:03.178926 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf968cd-d49b-483d-b0ff-f8b2d4721126 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f798123adf849b7915b68e477bbf546)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:03.181018 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf968cd-d49b-483d-b0ff-f8b2d4721126 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:03.335913 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-fbf968cd-d49b-483d-b0ff-f8b2d4721126 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f798123adf849b7915b68e477bbf546', 'request_id': 'req-fbf968cd-d49b-483d-b0ff-f8b2d4721126'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:03.347783 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-fbf968cd-d49b-483d-b0ff-f8b2d4721126 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f798123adf849b7915b68e477bbf546', 'request_id': 'req-fbf968cd-d49b-483d-b0ff-f8b2d4721126'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:03.363112 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-fbf968cd-d49b-483d-b0ff-f8b2d4721126 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f798123adf849b7915b68e477bbf546', 'request_id': 'req-fbf968cd-d49b-483d-b0ff-f8b2d4721126'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:03.381768 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-fbf968cd-d49b-483d-b0ff-f8b2d4721126 admin admin] The token cache is being invalidate because project 3f798123adf849b7915b68e477bbf546 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:13:03.390724 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-fbf968cd-d49b-483d-b0ff-f8b2d4721126 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:03.390724 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-fbf968cd-d49b-483d-b0ff-f8b2d4721126 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:03.401762 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 848/1707] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 9 12:13:02 2022] DELETE /identity/v3/projects/3f798123adf849b7915b68e477bbf546 => generated 0 bytes in 553 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:13:03.404055 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4404a8-8ad5-46cd-b6b8-ba0ebf464a0f None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:03.404660 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4404a8-8ad5-46cd-b6b8-ba0ebf464a0f None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:03.405280 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4404a8-8ad5-46cd-b6b8-ba0ebf464a0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:03.458366 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-de4404a8-8ad5-46cd-b6b8-ba0ebf464a0f None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:13:03.460094 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-de4404a8-8ad5-46cd-b6b8-ba0ebf464a0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:03.506024 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf968cd-d49b-483d-b0ff-f8b2d4721126 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:03.508650 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-fbf968cd-d49b-483d-b0ff-f8b2d4721126 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:03.673581 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:03.680169 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:03.782746 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf968cd-d49b-483d-b0ff-f8b2d4721126 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:03.804745 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d4b8df-6247-47e3-94a3-678497db65e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:03.808287 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d4b8df-6247-47e3-94a3-678497db65e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:03.809223 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d4b8df-6247-47e3-94a3-678497db65e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:03.811417 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d4b8df-6247-47e3-94a3-678497db65e3 admin admin] PATH_INFO: `/v3/users/3360a42d91bb4360b9ec0b02a4a91629` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:03.816222 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 860/1708] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:13:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:13:03.816453 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d4b8df-6247-47e3-94a3-678497db65e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3360a42d91bb4360b9ec0b02a4a91629)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:03.824489 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7ddb6d-e5b9-468b-be1f-64d5bc56cc57 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:03.824975 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7ddb6d-e5b9-468b-be1f-64d5bc56cc57 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:03.825421 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7ddb6d-e5b9-468b-be1f-64d5bc56cc57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:03.825929 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d4b8df-6247-47e3-94a3-678497db65e3 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:03.927681 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-ca7ddb6d-e5b9-468b-be1f-64d5bc56cc57 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:13:03.938751 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-ca7ddb6d-e5b9-468b-be1f-64d5bc56cc57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:03.974591 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 861/1709] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:13:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:13:03.977240 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7ddb6d-e5b9-468b-be1f-64d5bc56cc57 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:03.995260 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-19d4b8df-6247-47e3-94a3-678497db65e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3360a42d91bb4360b9ec0b02a4a91629'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:04.018512 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7ddb6d-e5b9-468b-be1f-64d5bc56cc57 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:04.047558 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cc3dec-25c5-4f0c-b46d-78a5625767db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:04.048375 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cc3dec-25c5-4f0c-b46d-78a5625767db admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:04.048814 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cc3dec-25c5-4f0c-b46d-78a5625767db admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:04.052826 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cc3dec-25c5-4f0c-b46d-78a5625767db admin admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:04.053777 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cc3dec-25c5-4f0c-b46d-78a5625767db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:13:04.054226 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cc3dec-25c5-4f0c-b46d-78a5625767db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:04.056746 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cc3dec-25c5-4f0c-b46d-78a5625767db admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:04.088374 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-19d4b8df-6247-47e3-94a3-678497db65e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3360a42d91bb4360b9ec0b02a4a91629'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:04.110036 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 862/1710] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 9 12:13:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:13:04.114954 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cc3dec-25c5-4f0c-b46d-78a5625767db admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:04.134777 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-19d4b8df-6247-47e3-94a3-678497db65e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3360a42d91bb4360b9ec0b02a4a91629'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:04.138207 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cc3dec-25c5-4f0c-b46d-78a5625767db admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:04.153970 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabc68ad-845a-4cf7-8047-72628994f82b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:04.154872 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabc68ad-845a-4cf7-8047-72628994f82b admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:04.155337 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabc68ad-845a-4cf7-8047-72628994f82b admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:04.155756 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabc68ad-845a-4cf7-8047-72628994f82b admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:04.156540 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabc68ad-845a-4cf7-8047-72628994f82b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:04.158803 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabc68ad-845a-4cf7-8047-72628994f82b admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:04.160960 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-19d4b8df-6247-47e3-94a3-678497db65e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3360a42d91bb4360b9ec0b02a4a91629'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:04.161802 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-19d4b8df-6247-47e3-94a3-678497db65e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3360a42d91bb4360b9ec0b02a4a91629'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:04.176053 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 849/1711] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 9 12:13:03 2022] DELETE /identity/v3/users/3360a42d91bb4360b9ec0b02a4a91629 => generated 0 bytes in 666 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:13:04.178417 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aced6c-64f7-4b7b-9098-d3d67c765a10 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:04.178704 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aced6c-64f7-4b7b-9098-d3d67c765a10 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:04.179020 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aced6c-64f7-4b7b-9098-d3d67c765a10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:04.227192 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 863/1712] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 9 12:13:04 2022] POST /identity/v3/projects => generated 369 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:13:04.231858 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabc68ad-845a-4cf7-8047-72628994f82b admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:04.235085 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-eabc68ad-845a-4cf7-8047-72628994f82b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:04.266704 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-71aced6c-64f7-4b7b-9098-d3d67c765a10 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:13:04.276809 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-71aced6c-64f7-4b7b-9098-d3d67c765a10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:04.465948 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabc68ad-845a-4cf7-8047-72628994f82b admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:04.482386 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de528f9a-3cea-49cf-9ac9-d46fbce5c093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:04.483079 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de528f9a-3cea-49cf-9ac9-d46fbce5c093 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:04.483500 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de528f9a-3cea-49cf-9ac9-d46fbce5c093 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:04.483794 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de528f9a-3cea-49cf-9ac9-d46fbce5c093 admin admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:04.504831 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de528f9a-3cea-49cf-9ac9-d46fbce5c093 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:04.506990 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de528f9a-3cea-49cf-9ac9-d46fbce5c093 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:04.607526 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 864/1713] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:13:04 2022] POST /identity/v3/users => generated 285 bytes in 374 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:13:04.608523 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de528f9a-3cea-49cf-9ac9-d46fbce5c093 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:04.635441 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 850/1714] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:13:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 457 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:13:04.646731 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de528f9a-3cea-49cf-9ac9-d46fbce5c093 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:04.658421 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dc4aa1-a8e8-4125-9c2d-b7ba49f76f26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:04.659474 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dc4aa1-a8e8-4125-9c2d-b7ba49f76f26 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:04.660135 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dc4aa1-a8e8-4125-9c2d-b7ba49f76f26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:04.660590 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dc4aa1-a8e8-4125-9c2d-b7ba49f76f26 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:04.661579 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dc4aa1-a8e8-4125-9c2d-b7ba49f76f26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:04.663908 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dc4aa1-a8e8-4125-9c2d-b7ba49f76f26 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:04.683160 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aced6c-64f7-4b7b-9098-d3d67c765a10 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:04.715231 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 865/1715] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:13:04 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:13:04.745951 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aced6c-64f7-4b7b-9098-d3d67c765a10 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:04.766168 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40ce254-cad5-428d-bfbb-ab2d49be7487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:04.766842 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40ce254-cad5-428d-bfbb-ab2d49be7487 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:04.767233 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40ce254-cad5-428d-bfbb-ab2d49be7487 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:04.767436 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40ce254-cad5-428d-bfbb-ab2d49be7487 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:04.768137 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40ce254-cad5-428d-bfbb-ab2d49be7487 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:04.770054 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40ce254-cad5-428d-bfbb-ab2d49be7487 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:04.834084 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dc4aa1-a8e8-4125-9c2d-b7ba49f76f26 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:04.840593 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 851/1716] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:13:04 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:13:04.860559 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dc4aa1-a8e8-4125-9c2d-b7ba49f76f26 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:04.868729 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2a46a2-3e18-4f2f-b9c0-fa90c1a60972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:04.870490 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2a46a2-3e18-4f2f-b9c0-fa90c1a60972 admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:04.874280 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2a46a2-3e18-4f2f-b9c0-fa90c1a60972 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:04.874832 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2a46a2-3e18-4f2f-b9c0-fa90c1a60972 admin admin] PATH_INFO: `/v3/projects/11c6b0f334604ca2b1a97ef326faafb8/users/e7f5f90833a94eb5999ebfcd8cd634f2/roles/eecd2f160dd844f5906f3d345780c501` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:04.924986 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40ce254-cad5-428d-bfbb-ab2d49be7487 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:04.927067 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-e40ce254-cad5-428d-bfbb-ab2d49be7487 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:04.949217 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2a46a2-3e18-4f2f-b9c0-fa90c1a60972 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11c6b0f334604ca2b1a97ef326faafb8, user_id=e7f5f90833a94eb5999ebfcd8cd634f2, role_id=eecd2f160dd844f5906f3d345780c501)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:04.958216 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2a46a2-3e18-4f2f-b9c0-fa90c1a60972 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:05.039687 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 866/1717] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:13:04 2022] PUT /identity/v3/projects/11c6b0f334604ca2b1a97ef326faafb8/users/e7f5f90833a94eb5999ebfcd8cd634f2/roles/eecd2f160dd844f5906f3d345780c501 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:13:05.051293 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc39ee46-9400-4973-bd00-dbbfabbf438a None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:05.051586 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc39ee46-9400-4973-bd00-dbbfabbf438a None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:05.053126 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc39ee46-9400-4973-bd00-dbbfabbf438a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:05.158487 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-fc39ee46-9400-4973-bd00-dbbfabbf438a None None] MFA Rules not processed for user `e7f5f90833a94eb5999ebfcd8cd634f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:13:05.160933 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-fc39ee46-9400-4973-bd00-dbbfabbf438a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:05.172503 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40ce254-cad5-428d-bfbb-ab2d49be7487 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:05.238494 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:05.391530 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:05.419155 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498765df-dd7c-4737-99e7-b43a87c6deaa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:05.420918 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498765df-dd7c-4737-99e7-b43a87c6deaa service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:05.421215 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498765df-dd7c-4737-99e7-b43a87c6deaa service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:05.422360 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498765df-dd7c-4737-99e7-b43a87c6deaa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:05.439744 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 867/1718] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:13:05 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 399 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:13:05.444714 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc39ee46-9400-4973-bd00-dbbfabbf438a None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:05.623417 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498765df-dd7c-4737-99e7-b43a87c6deaa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:05.625593 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498765df-dd7c-4737-99e7-b43a87c6deaa service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:05.640444 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 852/1719] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:13:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 716 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:13:05.641142 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc39ee46-9400-4973-bd00-dbbfabbf438a None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:05.657844 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fb6ac7-f460-41ab-9430-b6a790b813f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:05.658069 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fb6ac7-f460-41ab-9430-b6a790b813f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:05.658357 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fb6ac7-f460-41ab-9430-b6a790b813f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:05.658632 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fb6ac7-f460-41ab-9430-b6a790b813f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:05.659358 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fb6ac7-f460-41ab-9430-b6a790b813f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:05.668306 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fb6ac7-f460-41ab-9430-b6a790b813f5 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:05.746953 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 868/1720] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 9 12:13:05 2022] POST /identity/v3/projects => generated 367 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:13:05.748567 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498765df-dd7c-4737-99e7-b43a87c6deaa service neutron] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:05.763249 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498765df-dd7c-4737-99e7-b43a87c6deaa service neutron] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:06.112820 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92440ff5-3bb5-46ad-9d1f-6ecda7ff4f42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:06.113450 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92440ff5-3bb5-46ad-9d1f-6ecda7ff4f42 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:06.113730 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92440ff5-3bb5-46ad-9d1f-6ecda7ff4f42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:06.114002 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92440ff5-3bb5-46ad-9d1f-6ecda7ff4f42 admin admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:06.114791 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92440ff5-3bb5-46ad-9d1f-6ecda7ff4f42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:06.116945 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92440ff5-3bb5-46ad-9d1f-6ecda7ff4f42 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:06.173063 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 853/1721] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:13:05 2022] POST /identity/v3/users => generated 290 bytes in 426 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:13:06.177180 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fb6ac7-f460-41ab-9430-b6a790b813f5 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:06.197348 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fb6ac7-f460-41ab-9430-b6a790b813f5 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:06.220607 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22041750-ad2f-4735-a6f3-f47e11761b83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:06.221796 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22041750-ad2f-4735-a6f3-f47e11761b83 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:06.223294 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22041750-ad2f-4735-a6f3-f47e11761b83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:06.224489 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22041750-ad2f-4735-a6f3-f47e11761b83 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:06.228655 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22041750-ad2f-4735-a6f3-f47e11761b83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:06.233570 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22041750-ad2f-4735-a6f3-f47e11761b83 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:06.267818 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 869/1722] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:13:06 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:13:06.272291 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92440ff5-3bb5-46ad-9d1f-6ecda7ff4f42 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:06.302430 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92440ff5-3bb5-46ad-9d1f-6ecda7ff4f42 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:06.326440 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e08a3b0-3ee1-4f01-8cd7-7e56c0e1dfb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:06.327184 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e08a3b0-3ee1-4f01-8cd7-7e56c0e1dfb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:06.327523 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e08a3b0-3ee1-4f01-8cd7-7e56c0e1dfb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:06.327812 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e08a3b0-3ee1-4f01-8cd7-7e56c0e1dfb6 admin admin] PATH_INFO: `/v3/projects/81bc85bbb7bc41498fa58b25f0ab1309/users/030e1b15a9d449ff8ed811c897c23350/roles/3f2ca33902a04b6dbc2ef7d06d7289d3` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:06.394326 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e08a3b0-3ee1-4f01-8cd7-7e56c0e1dfb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81bc85bbb7bc41498fa58b25f0ab1309, user_id=030e1b15a9d449ff8ed811c897c23350, role_id=3f2ca33902a04b6dbc2ef7d06d7289d3)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:06.397645 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e08a3b0-3ee1-4f01-8cd7-7e56c0e1dfb6 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:06.444059 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 854/1723] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:13:06 2022] PUT /identity/v3/projects/81bc85bbb7bc41498fa58b25f0ab1309/users/030e1b15a9d449ff8ed811c897c23350/roles/3f2ca33902a04b6dbc2ef7d06d7289d3 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:13:06.448612 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22041750-ad2f-4735-a6f3-f47e11761b83 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:06.465098 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22041750-ad2f-4735-a6f3-f47e11761b83 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:06.720349 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c74d677-75a0-46d0-81d0-462c4352a7f3 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:06.720770 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c74d677-75a0-46d0-81d0-462c4352a7f3 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:06.721116 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c74d677-75a0-46d0-81d0-462c4352a7f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:06.795180 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db96112-47ef-461e-b086-1b3c824a62c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:06.795874 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db96112-47ef-461e-b086-1b3c824a62c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:06.796163 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db96112-47ef-461e-b086-1b3c824a62c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:06.796443 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db96112-47ef-461e-b086-1b3c824a62c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:06.798036 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db96112-47ef-461e-b086-1b3c824a62c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:06.822374 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db96112-47ef-461e-b086-1b3c824a62c2 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:06.862608 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-9c74d677-75a0-46d0-81d0-462c4352a7f3 None None] MFA Rules not processed for user `b9a5c9fb3f664ae097f0c3ce6f6c0764`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:13:06.863919 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-9c74d677-75a0-46d0-81d0-462c4352a7f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:06.865774 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 870/1724] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:13:06 2022] GET /identity/v3/roles => generated 1420 bytes in 418 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 12:13:06.867394 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b88007-8ef6-459b-bbee-1a2d7f35f81e None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:06.867683 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b88007-8ef6-459b-bbee-1a2d7f35f81e None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:06.867956 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b88007-8ef6-459b-bbee-1a2d7f35f81e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:06.943766 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:06.960417 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-e2b88007-8ef6-459b-bbee-1a2d7f35f81e None None] MFA Rules not processed for user `992de6f86b42495dafe73cb2c393fb4e`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:13:06.964407 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-e2b88007-8ef6-459b-bbee-1a2d7f35f81e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:07.254014 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 855/1725] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:13:06 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 535 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:13:07.255043 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c74d677-75a0-46d0-81d0-462c4352a7f3 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:07.284480 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c74d677-75a0-46d0-81d0-462c4352a7f3 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:07.289045 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 871/1726] 10.222.0.71 () {58 vars in 1015 bytes} [Fri Sep 9 12:13:06 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 423 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:13:07.290780 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b88007-8ef6-459b-bbee-1a2d7f35f81e None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:07.292081 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e0a3c3-7c18-4300-8f33-d73c5f5c4356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:07.293212 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e0a3c3-7c18-4300-8f33-d73c5f5c4356 admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:07.293786 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e0a3c3-7c18-4300-8f33-d73c5f5c4356 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:07.294306 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e0a3c3-7c18-4300-8f33-d73c5f5c4356 admin admin] PATH_INFO: `/v3/projects/81bc85bbb7bc41498fa58b25f0ab1309/users/030e1b15a9d449ff8ed811c897c23350/roles/3f2ca33902a04b6dbc2ef7d06d7289d3` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:07.294551 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-e2b88007-8ef6-459b-bbee-1a2d7f35f81e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:07.297488 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e0a3c3-7c18-4300-8f33-d73c5f5c4356 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81bc85bbb7bc41498fa58b25f0ab1309, user_id=030e1b15a9d449ff8ed811c897c23350, role_id=3f2ca33902a04b6dbc2ef7d06d7289d3)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:07.300944 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e0a3c3-7c18-4300-8f33-d73c5f5c4356 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:07.318161 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 856/1727] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:13:07 2022] PUT /identity/v3/projects/81bc85bbb7bc41498fa58b25f0ab1309/users/030e1b15a9d449ff8ed811c897c23350/roles/3f2ca33902a04b6dbc2ef7d06d7289d3 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:13:07.329417 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e0a3c3-7c18-4300-8f33-d73c5f5c4356 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:07.358848 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b88007-8ef6-459b-bbee-1a2d7f35f81e None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:07.363010 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:07.451537 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e0a3c3-7c18-4300-8f33-d73c5f5c4356 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:07.456380 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6e2015-91d6-48a4-ad63-e9dbc93470c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:07.460947 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6e2015-91d6-48a4-ad63-e9dbc93470c0 service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:07.461246 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6e2015-91d6-48a4-ad63-e9dbc93470c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:07.461579 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6e2015-91d6-48a4-ad63-e9dbc93470c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:07.482259 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6e2015-91d6-48a4-ad63-e9dbc93470c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:07.485947 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6e2015-91d6-48a4-ad63-e9dbc93470c0 service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:07.515368 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 857/1728] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:13:07 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:13:07.523166 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ecbd64-df9d-4471-974e-355429048c8c None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:07.523451 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ecbd64-df9d-4471-974e-355429048c8c None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:07.523872 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ecbd64-df9d-4471-974e-355429048c8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:07.525467 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb5feaa-fc7f-469f-a801-8ea1084d4cbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:07.526664 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb5feaa-fc7f-469f-a801-8ea1084d4cbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:07.527022 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb5feaa-fc7f-469f-a801-8ea1084d4cbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:07.528559 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb5feaa-fc7f-469f-a801-8ea1084d4cbd admin admin] PATH_INFO: `/v3/projects/977164fbbeac4040a6d2a7c965ea1d9a` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:07.534355 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb5feaa-fc7f-469f-a801-8ea1084d4cbd admin admin] RBAC: Authorizing `identity:delete_project(project_id=977164fbbeac4040a6d2a7c965ea1d9a)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:07.541195 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb5feaa-fc7f-469f-a801-8ea1084d4cbd admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:07.616305 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-22ecbd64-df9d-4471-974e-355429048c8c None None] MFA Rules not processed for user `030e1b15a9d449ff8ed811c897c23350`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:13:07.618820 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-22ecbd64-df9d-4471-974e-355429048c8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:07.702296 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-2eb5feaa-fc7f-469f-a801-8ea1084d4cbd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '977164fbbeac4040a6d2a7c965ea1d9a', 'request_id': 'req-2eb5feaa-fc7f-469f-a801-8ea1084d4cbd'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:07.716247 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-2eb5feaa-fc7f-469f-a801-8ea1084d4cbd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '977164fbbeac4040a6d2a7c965ea1d9a', 'request_id': 'req-2eb5feaa-fc7f-469f-a801-8ea1084d4cbd'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:07.732229 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-2eb5feaa-fc7f-469f-a801-8ea1084d4cbd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '977164fbbeac4040a6d2a7c965ea1d9a', 'request_id': 'req-2eb5feaa-fc7f-469f-a801-8ea1084d4cbd'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:07.763557 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-2eb5feaa-fc7f-469f-a801-8ea1084d4cbd admin admin] The token cache is being invalidate because project 977164fbbeac4040a6d2a7c965ea1d9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81591) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:13:07.763895 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-2eb5feaa-fc7f-469f-a801-8ea1084d4cbd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:07.764637 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-2eb5feaa-fc7f-469f-a801-8ea1084d4cbd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:07.774323 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 872/1729] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 9 12:13:07 2022] DELETE /identity/v3/projects/977164fbbeac4040a6d2a7c965ea1d9a => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:13:07.778023 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fcca86-698f-4386-906d-9383dc4bb4b3 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:07.778313 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fcca86-698f-4386-906d-9383dc4bb4b3 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:07.778591 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fcca86-698f-4386-906d-9383dc4bb4b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:07.831543 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-96fcca86-698f-4386-906d-9383dc4bb4b3 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:13:07.832904 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-96fcca86-698f-4386-906d-9383dc4bb4b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:07.954619 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 858/1730] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:13:07 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 439 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:13:07.963259 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ecbd64-df9d-4471-974e-355429048c8c None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:07.964942 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-22ecbd64-df9d-4471-974e-355429048c8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:08.026720 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 873/1731] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:13:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:13:08.028567 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c179364-bdc2-46a8-babe-e6f2f4b90431 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:08.029118 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c179364-bdc2-46a8-babe-e6f2f4b90431 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:08.029597 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c179364-bdc2-46a8-babe-e6f2f4b90431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:08.053462 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-3c179364-bdc2-46a8-babe-e6f2f4b90431 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:13:08.055511 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-3c179364-bdc2-46a8-babe-e6f2f4b90431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:08.084715 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 874/1732] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:13:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:13:08.085248 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c179364-bdc2-46a8-babe-e6f2f4b90431 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:08.092248 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-3c179364-bdc2-46a8-babe-e6f2f4b90431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:08.127064 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:08.181720 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ecbd64-df9d-4471-974e-355429048c8c None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:08.231829 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:08.352054 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c179364-bdc2-46a8-babe-e6f2f4b90431 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:08.361017 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8d175e-885c-4a05-9421-a6ca31fa8ffe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535c258100df40b58a5ee44b2c21d621', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:08.364743 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8d175e-885c-4a05-9421-a6ca31fa8ffe service cinder] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:08.365051 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8d175e-885c-4a05-9421-a6ca31fa8ffe service cinder] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:08.365860 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c189636-6ea8-45d5-ba3a-8edf284a30e0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70bac169d5cd43e1926f310344ac11fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:08.366175 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8d175e-885c-4a05-9421-a6ca31fa8ffe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:08.366823 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c189636-6ea8-45d5-ba3a-8edf284a30e0 service nova] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:08.367887 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c189636-6ea8-45d5-ba3a-8edf284a30e0 service nova] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:08.368313 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c189636-6ea8-45d5-ba3a-8edf284a30e0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:08.370612 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-8c189636-6ea8-45d5-ba3a-8edf284a30e0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:08.568370 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c189636-6ea8-45d5-ba3a-8edf284a30e0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:08.575412 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c189636-6ea8-45d5-ba3a-8edf284a30e0 service nova] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:08.611712 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 875/1733] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:13:08 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 525 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:13:08.613003 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c189636-6ea8-45d5-ba3a-8edf284a30e0 service nova] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:08.618863 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8d175e-885c-4a05-9421-a6ca31fa8ffe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:08.628200 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8d175e-885c-4a05-9421-a6ca31fa8ffe service cinder] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:08.664828 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c189636-6ea8-45d5-ba3a-8edf284a30e0 service nova] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:08.672370 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 859/1734] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:13:07 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 710 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:13:08.675159 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8d175e-885c-4a05-9421-a6ca31fa8ffe service cinder] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:08.679496 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c83778-b05b-41f4-9cf8-ae2c2aaa6da0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:08.680837 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-ed8d175e-885c-4a05-9421-a6ca31fa8ffe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:08.684116 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c83778-b05b-41f4-9cf8-ae2c2aaa6da0 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:08.685663 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c83778-b05b-41f4-9cf8-ae2c2aaa6da0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:08.688133 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c83778-b05b-41f4-9cf8-ae2c2aaa6da0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:08.688133 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c83778-b05b-41f4-9cf8-ae2c2aaa6da0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:13:08.688133 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c83778-b05b-41f4-9cf8-ae2c2aaa6da0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:08.698217 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c83778-b05b-41f4-9cf8-ae2c2aaa6da0 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:08.732321 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8d175e-885c-4a05-9421-a6ca31fa8ffe service cinder] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:08.747097 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 876/1735] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 9 12:13:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:13:08.748516 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c83778-b05b-41f4-9cf8-ae2c2aaa6da0 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:08.757317 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a574b1ca-b608-4d51-93bc-91d5fde397f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:08.758156 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a574b1ca-b608-4d51-93bc-91d5fde397f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:08.760763 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a574b1ca-b608-4d51-93bc-91d5fde397f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:08.761158 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a574b1ca-b608-4d51-93bc-91d5fde397f8 admin admin] PATH_INFO: `/v3/projects/73d7d7afa06b4426970d12058524f938` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:08.771108 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a574b1ca-b608-4d51-93bc-91d5fde397f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=73d7d7afa06b4426970d12058524f938)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:08.776080 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a574b1ca-b608-4d51-93bc-91d5fde397f8 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:08.782065 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c83778-b05b-41f4-9cf8-ae2c2aaa6da0 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:08.800042 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffe44cc-95a4-4364-bc42-1f9073494c11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:08.800831 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffe44cc-95a4-4364-bc42-1f9073494c11 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:08.801425 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffe44cc-95a4-4364-bc42-1f9073494c11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:08.802026 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffe44cc-95a4-4364-bc42-1f9073494c11 admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:08.803698 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffe44cc-95a4-4364-bc42-1f9073494c11 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:08.806098 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffe44cc-95a4-4364-bc42-1f9073494c11 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:08.893601 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 877/1736] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 9 12:13:08 2022] POST /identity/v3/projects => generated 401 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:13:08.897255 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffe44cc-95a4-4364-bc42-1f9073494c11 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:08.942293 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-a574b1ca-b608-4d51-93bc-91d5fde397f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73d7d7afa06b4426970d12058524f938', 'request_id': 'req-a574b1ca-b608-4d51-93bc-91d5fde397f8'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:08.972663 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-a574b1ca-b608-4d51-93bc-91d5fde397f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73d7d7afa06b4426970d12058524f938', 'request_id': 'req-a574b1ca-b608-4d51-93bc-91d5fde397f8'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:09.000112 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-a574b1ca-b608-4d51-93bc-91d5fde397f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73d7d7afa06b4426970d12058524f938', 'request_id': 'req-a574b1ca-b608-4d51-93bc-91d5fde397f8'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:09.043282 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-a574b1ca-b608-4d51-93bc-91d5fde397f8 admin admin] The token cache is being invalidate because project 73d7d7afa06b4426970d12058524f938 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:13:09.043744 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-a574b1ca-b608-4d51-93bc-91d5fde397f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:09.045601 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-a574b1ca-b608-4d51-93bc-91d5fde397f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:09.055146 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 860/1737] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 9 12:13:08 2022] DELETE /identity/v3/projects/73d7d7afa06b4426970d12058524f938 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:13:09.056278 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a574b1ca-b608-4d51-93bc-91d5fde397f8 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:09.071466 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-a574b1ca-b608-4d51-93bc-91d5fde397f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:09.197167 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:09.197368 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffe44cc-95a4-4364-bc42-1f9073494c11 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:09.217749 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168f8c02-fa78-4567-ba13-07dde016372f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:09.220028 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168f8c02-fa78-4567-ba13-07dde016372f admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:09.223076 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168f8c02-fa78-4567-ba13-07dde016372f admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:09.223505 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168f8c02-fa78-4567-ba13-07dde016372f admin admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:09.224792 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168f8c02-fa78-4567-ba13-07dde016372f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:09.228603 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168f8c02-fa78-4567-ba13-07dde016372f admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:09.308297 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 878/1738] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:13:08 2022] POST /identity/v3/users => generated 301 bytes in 414 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:13:09.315525 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168f8c02-fa78-4567-ba13-07dde016372f admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:09.326815 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-168f8c02-fa78-4567-ba13-07dde016372f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:09.346169 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a574b1ca-b608-4d51-93bc-91d5fde397f8 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:09.380772 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168f8c02-fa78-4567-ba13-07dde016372f admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:09.382559 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d8efa2-b8a7-4593-85fc-8ed260b61932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:09.387341 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d8efa2-b8a7-4593-85fc-8ed260b61932 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:09.389271 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d8efa2-b8a7-4593-85fc-8ed260b61932 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:09.389919 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d8efa2-b8a7-4593-85fc-8ed260b61932 admin admin] PATH_INFO: `/v3/users/2e7306305f884561bc053ab70e63570f` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:09.392668 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d8efa2-b8a7-4593-85fc-8ed260b61932 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e7306305f884561bc053ab70e63570f)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:09.395348 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d8efa2-b8a7-4593-85fc-8ed260b61932 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:09.410829 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0cae0a-1f97-4e8c-b808-a561d022bafe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:09.411614 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0cae0a-1f97-4e8c-b808-a561d022bafe admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:09.412579 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0cae0a-1f97-4e8c-b808-a561d022bafe admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:09.414235 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0cae0a-1f97-4e8c-b808-a561d022bafe admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:09.417476 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0cae0a-1f97-4e8c-b808-a561d022bafe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:09.420574 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0cae0a-1f97-4e8c-b808-a561d022bafe admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:09.465654 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 879/1739] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:13:09 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:13:09.466735 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0cae0a-1f97-4e8c-b808-a561d022bafe admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:09.504646 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0cae0a-1f97-4e8c-b808-a561d022bafe admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:09.519562 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d7a769-741d-4ff8-8a43-2677895d9262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:09.520312 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d7a769-741d-4ff8-8a43-2677895d9262 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:09.520942 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d7a769-741d-4ff8-8a43-2677895d9262 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:09.522394 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d7a769-741d-4ff8-8a43-2677895d9262 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:09.524701 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d7a769-741d-4ff8-8a43-2677895d9262 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:09.535442 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d7a769-741d-4ff8-8a43-2677895d9262 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:09.552062 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:09.576002 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-e1d8efa2-b8a7-4593-85fc-8ed260b61932 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e7306305f884561bc053ab70e63570f'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:09.579860 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 880/1740] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:13:09 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:13:09.592222 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d7a769-741d-4ff8-8a43-2677895d9262 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:09.615656 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d7a769-741d-4ff8-8a43-2677895d9262 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:09.624364 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-e1d8efa2-b8a7-4593-85fc-8ed260b61932 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e7306305f884561bc053ab70e63570f'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:09.640161 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3bc6cf-5d1c-4d38-9f02-eeb2cb2f6475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:09.641186 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3bc6cf-5d1c-4d38-9f02-eeb2cb2f6475 admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:09.642677 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3bc6cf-5d1c-4d38-9f02-eeb2cb2f6475 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:09.643178 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3bc6cf-5d1c-4d38-9f02-eeb2cb2f6475 admin admin] PATH_INFO: `/v3/projects/d9613b08ac104a45b9796d38a180839a/users/c07d3a751567467ea4fd94334f1a96ca/roles/eecd2f160dd844f5906f3d345780c501` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:09.652673 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-e1d8efa2-b8a7-4593-85fc-8ed260b61932 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e7306305f884561bc053ab70e63570f'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:09.677211 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-e1d8efa2-b8a7-4593-85fc-8ed260b61932 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e7306305f884561bc053ab70e63570f'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:09.678318 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-e1d8efa2-b8a7-4593-85fc-8ed260b61932 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e7306305f884561bc053ab70e63570f'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:09.699496 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 861/1741] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 9 12:13:09 2022] DELETE /identity/v3/users/2e7306305f884561bc053ab70e63570f => generated 0 bytes in 637 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:13:09.704013 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3bc6cf-5d1c-4d38-9f02-eeb2cb2f6475 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9613b08ac104a45b9796d38a180839a, user_id=c07d3a751567467ea4fd94334f1a96ca, role_id=eecd2f160dd844f5906f3d345780c501)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:09.707462 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3bc6cf-5d1c-4d38-9f02-eeb2cb2f6475 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:09.750098 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 881/1742] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:13:09 2022] PUT /identity/v3/projects/d9613b08ac104a45b9796d38a180839a/users/c07d3a751567467ea4fd94334f1a96ca/roles/eecd2f160dd844f5906f3d345780c501 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:13:09.754786 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c22652-35bf-4e89-9dd1-5e9e503165ec None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:09.755211 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c22652-35bf-4e89-9dd1-5e9e503165ec None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:09.755590 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c22652-35bf-4e89-9dd1-5e9e503165ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:09.822390 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-d0c22652-35bf-4e89-9dd1-5e9e503165ec None None] MFA Rules not processed for user `c07d3a751567467ea4fd94334f1a96ca`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:13:09.824388 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-d0c22652-35bf-4e89-9dd1-5e9e503165ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:09.973506 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3bc6cf-5d1c-4d38-9f02-eeb2cb2f6475 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:09.975446 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-3d3bc6cf-5d1c-4d38-9f02-eeb2cb2f6475 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:09.977610 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 862/1743] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:13:09 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:13:10.137288 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3bc6cf-5d1c-4d38-9f02-eeb2cb2f6475 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:10.152782 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff5b375-5ca6-4b6c-a62c-dd9dbf7e3872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:10.153540 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff5b375-5ca6-4b6c-a62c-dd9dbf7e3872 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:10.154100 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff5b375-5ca6-4b6c-a62c-dd9dbf7e3872 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:10.154519 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff5b375-5ca6-4b6c-a62c-dd9dbf7e3872 admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:10.155222 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff5b375-5ca6-4b6c-a62c-dd9dbf7e3872 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:10.157448 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff5b375-5ca6-4b6c-a62c-dd9dbf7e3872 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:10.222598 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c22652-35bf-4e89-9dd1-5e9e503165ec None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:10.224536 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 882/1744] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 9 12:13:09 2022] POST /identity/v3/projects => generated 401 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:13:10.250619 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c22652-35bf-4e89-9dd1-5e9e503165ec None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:10.315126 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:10.390660 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e984b91-90f3-4cc2-a6aa-2bc917dee917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:10.391601 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e984b91-90f3-4cc2-a6aa-2bc917dee917 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:10.392105 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e984b91-90f3-4cc2-a6aa-2bc917dee917 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:10.392562 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e984b91-90f3-4cc2-a6aa-2bc917dee917 admin admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:10.393560 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e984b91-90f3-4cc2-a6aa-2bc917dee917 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:10.396116 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e984b91-90f3-4cc2-a6aa-2bc917dee917 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:10.450857 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 863/1745] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:13:10 2022] POST /identity/v3/users => generated 307 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:13:10.455293 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff5b375-5ca6-4b6c-a62c-dd9dbf7e3872 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:10.473663 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff5b375-5ca6-4b6c-a62c-dd9dbf7e3872 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:10.489977 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca409a01-972c-49d3-a836-68702da78596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:10.490799 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca409a01-972c-49d3-a836-68702da78596 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:10.491172 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca409a01-972c-49d3-a836-68702da78596 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:10.491517 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca409a01-972c-49d3-a836-68702da78596 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:10.492344 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca409a01-972c-49d3-a836-68702da78596 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:10.494053 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca409a01-972c-49d3-a836-68702da78596 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:10.524039 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e984b91-90f3-4cc2-a6aa-2bc917dee917 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:10.536329 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 883/1746] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:13:10 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:13:10.542510 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e984b91-90f3-4cc2-a6aa-2bc917dee917 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:10.560597 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5a3a77-6b77-43e9-b109-fd70783beb8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:10.561835 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5a3a77-6b77-43e9-b109-fd70783beb8c admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:10.562341 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5a3a77-6b77-43e9-b109-fd70783beb8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:10.562798 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5a3a77-6b77-43e9-b109-fd70783beb8c admin admin] PATH_INFO: `/v3/projects/cc342aa9bae54145bcdec06443c71380/users/1a936453f7734fb092d27deb0a3b1569/roles/3f2ca33902a04b6dbc2ef7d06d7289d3` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:10.600560 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5a3a77-6b77-43e9-b109-fd70783beb8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc342aa9bae54145bcdec06443c71380, user_id=1a936453f7734fb092d27deb0a3b1569, role_id=3f2ca33902a04b6dbc2ef7d06d7289d3)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:10.604997 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5a3a77-6b77-43e9-b109-fd70783beb8c admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:10.649118 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 864/1747] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:13:10 2022] PUT /identity/v3/projects/cc342aa9bae54145bcdec06443c71380/users/1a936453f7734fb092d27deb0a3b1569/roles/3f2ca33902a04b6dbc2ef7d06d7289d3 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:13:10.652005 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca409a01-972c-49d3-a836-68702da78596 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:10.665670 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca409a01-972c-49d3-a836-68702da78596 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:10.833491 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449305af-3679-40df-84d5-f58be865996c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:10.834235 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449305af-3679-40df-84d5-f58be865996c admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:10.834819 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449305af-3679-40df-84d5-f58be865996c admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:10.835215 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449305af-3679-40df-84d5-f58be865996c admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:10.836203 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449305af-3679-40df-84d5-f58be865996c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:10.838826 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449305af-3679-40df-84d5-f58be865996c admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:10.858951 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 884/1748] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:13:10 2022] GET /identity/v3/roles => generated 1420 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 12:13:10.862832 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5a3a77-6b77-43e9-b109-fd70783beb8c admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:10.883012 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5a3a77-6b77-43e9-b109-fd70783beb8c admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:10.889578 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ac9645-3fd1-41b7-b972-59c7f2c5f6f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:10.890694 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ac9645-3fd1-41b7-b972-59c7f2c5f6f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:10.890694 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ac9645-3fd1-41b7-b972-59c7f2c5f6f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:10.890932 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ac9645-3fd1-41b7-b972-59c7f2c5f6f2 admin admin] PATH_INFO: `/v3/projects/cc342aa9bae54145bcdec06443c71380/users/1a936453f7734fb092d27deb0a3b1569/roles/3f2ca33902a04b6dbc2ef7d06d7289d3` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:10.893174 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ac9645-3fd1-41b7-b972-59c7f2c5f6f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc342aa9bae54145bcdec06443c71380, user_id=1a936453f7734fb092d27deb0a3b1569, role_id=3f2ca33902a04b6dbc2ef7d06d7289d3)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:10.896073 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ac9645-3fd1-41b7-b972-59c7f2c5f6f2 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:10.948476 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 865/1749] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:13:10 2022] PUT /identity/v3/projects/cc342aa9bae54145bcdec06443c71380/users/1a936453f7734fb092d27deb0a3b1569/roles/3f2ca33902a04b6dbc2ef7d06d7289d3 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:13:10.969526 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ae2bf8-596b-4aa4-8110-87f6d0ef370f None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:10.970367 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ae2bf8-596b-4aa4-8110-87f6d0ef370f None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:10.970809 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ae2bf8-596b-4aa4-8110-87f6d0ef370f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:11.053666 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-d0ae2bf8-596b-4aa4-8110-87f6d0ef370f None None] MFA Rules not processed for user `1a936453f7734fb092d27deb0a3b1569`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:13:11.055054 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-d0ae2bf8-596b-4aa4-8110-87f6d0ef370f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:11.182611 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:11.282362 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 885/1750] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:13:10 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:13:11.337332 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ac9645-3fd1-41b7-b972-59c7f2c5f6f2 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:11.340032 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-76ac9645-3fd1-41b7-b972-59c7f2c5f6f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:11.520972 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ac9645-3fd1-41b7-b972-59c7f2c5f6f2 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:11.536194 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a622321-bef6-4fd8-a3cf-dc8fb6d47dcb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535c258100df40b58a5ee44b2c21d621', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:11.539445 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a622321-bef6-4fd8-a3cf-dc8fb6d47dcb service cinder] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:11.540309 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a622321-bef6-4fd8-a3cf-dc8fb6d47dcb service cinder] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:11.540862 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a622321-bef6-4fd8-a3cf-dc8fb6d47dcb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:11.595111 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a622321-bef6-4fd8-a3cf-dc8fb6d47dcb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:11.598628 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a622321-bef6-4fd8-a3cf-dc8fb6d47dcb service cinder] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:11.627507 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 866/1751] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:13:11 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:13:11.880516 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea89f50-d071-43f1-b4f3-1e435216b485 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:11.881225 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea89f50-d071-43f1-b4f3-1e435216b485 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:11.882468 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea89f50-d071-43f1-b4f3-1e435216b485 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:11.886919 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 886/1752] 10.222.0.49 () {54 vars in 921 bytes} [Fri Sep 9 12:13:11 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:13:11.901391 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bd0fa3-039c-4945-8241-d22bd3701eb8 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:11.901693 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bd0fa3-039c-4945-8241-d22bd3701eb8 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:11.901940 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bd0fa3-039c-4945-8241-d22bd3701eb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:11.936988 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-c5bd0fa3-039c-4945-8241-d22bd3701eb8 None None] MFA Rules not processed for user `70bac169d5cd43e1926f310344ac11fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:13:11.940765 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-c5bd0fa3-039c-4945-8241-d22bd3701eb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:11.949820 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea89f50-d071-43f1-b4f3-1e435216b485 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:11.966101 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea89f50-d071-43f1-b4f3-1e435216b485 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:11.978993 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3065e5d6-d24b-4631-9346-dcd0b9311df2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535c258100df40b58a5ee44b2c21d621', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:11.979916 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3065e5d6-d24b-4631-9346-dcd0b9311df2 service cinder] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:11.980204 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3065e5d6-d24b-4631-9346-dcd0b9311df2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:11.980475 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3065e5d6-d24b-4631-9346-dcd0b9311df2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:12.132989 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3065e5d6-d24b-4631-9346-dcd0b9311df2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:12.137516 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3065e5d6-d24b-4631-9346-dcd0b9311df2 service cinder] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:12.179046 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 867/1753] 10.222.0.49 () {58 vars in 1018 bytes} [Fri Sep 9 12:13:11 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:13:12.209861 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 887/1754] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:13:11 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:13:12.967606 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bd0fa3-039c-4945-8241-d22bd3701eb8 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:12.969826 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-c5bd0fa3-039c-4945-8241-d22bd3701eb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:12.992418 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bd0fa3-039c-4945-8241-d22bd3701eb8 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:13.020481 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb46c7c-b941-44df-a68c-37f19d91952f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70bac169d5cd43e1926f310344ac11fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:13.021348 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb46c7c-b941-44df-a68c-37f19d91952f service nova] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:13.021778 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb46c7c-b941-44df-a68c-37f19d91952f service nova] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:13.022197 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb46c7c-b941-44df-a68c-37f19d91952f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:13.040613 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb46c7c-b941-44df-a68c-37f19d91952f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:13.043484 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb46c7c-b941-44df-a68c-37f19d91952f service nova] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:13.057419 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 868/1755] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:13:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:13:13.281336 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3065e5d6-d24b-4631-9346-dcd0b9311df2 service cinder] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:13.282972 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-3065e5d6-d24b-4631-9346-dcd0b9311df2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:13.330986 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efd6839-2786-42b5-8a61-8650b8563294 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:13.331192 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efd6839-2786-42b5-8a61-8650b8563294 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:13.331643 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efd6839-2786-42b5-8a61-8650b8563294 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:13.333080 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 869/1756] 10.222.0.49 () {54 vars in 921 bytes} [Fri Sep 9 12:13:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:13:13.361937 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a253f168-d0cd-4d9e-9690-f8eedf17b667 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:13.362295 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a253f168-d0cd-4d9e-9690-f8eedf17b667 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:13.362599 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a253f168-d0cd-4d9e-9690-f8eedf17b667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:13.396092 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:13.414280 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3065e5d6-d24b-4631-9346-dcd0b9311df2 service cinder] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:13.425177 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25054ab5-6a79-4d28-91ef-91638a6c2544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:13.426570 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25054ab5-6a79-4d28-91ef-91638a6c2544 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:13.427379 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25054ab5-6a79-4d28-91ef-91638a6c2544 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:13.427905 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25054ab5-6a79-4d28-91ef-91638a6c2544 admin admin] PATH_INFO: `/v3/users/e7f5f90833a94eb5999ebfcd8cd634f2` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:13.440330 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-a253f168-d0cd-4d9e-9690-f8eedf17b667 None None] MFA Rules not processed for user `70bac169d5cd43e1926f310344ac11fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:13:13.440501 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25054ab5-6a79-4d28-91ef-91638a6c2544 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7f5f90833a94eb5999ebfcd8cd634f2)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:13.447292 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-a253f168-d0cd-4d9e-9690-f8eedf17b667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:13.450165 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25054ab5-6a79-4d28-91ef-91638a6c2544 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:13.453954 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 870/1757] 10.222.0.49 () {58 vars in 1018 bytes} [Fri Sep 9 12:13:13 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:13:13.455094 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a253f168-d0cd-4d9e-9690-f8eedf17b667 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:13.459863 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-a253f168-d0cd-4d9e-9690-f8eedf17b667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:13.484741 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a253f168-d0cd-4d9e-9690-f8eedf17b667 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:13.490335 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba0a83d-0c56-4d44-aea9-159b0a1b00d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:13.491124 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba0a83d-0c56-4d44-aea9-159b0a1b00d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:13.492413 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba0a83d-0c56-4d44-aea9-159b0a1b00d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:13.493059 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba0a83d-0c56-4d44-aea9-159b0a1b00d3 admin admin] PATH_INFO: `/v3/projects/4e5b3edbfe2743a0854e53923d366cf2` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:13.495133 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba0a83d-0c56-4d44-aea9-159b0a1b00d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e5b3edbfe2743a0854e53923d366cf2)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:13.497818 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba0a83d-0c56-4d44-aea9-159b0a1b00d3 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:13.540980 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-25054ab5-6a79-4d28-91ef-91638a6c2544 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7f5f90833a94eb5999ebfcd8cd634f2'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:13.550602 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-25054ab5-6a79-4d28-91ef-91638a6c2544 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7f5f90833a94eb5999ebfcd8cd634f2'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:13.565113 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-3ba0a83d-0c56-4d44-aea9-159b0a1b00d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e5b3edbfe2743a0854e53923d366cf2', 'request_id': 'req-3ba0a83d-0c56-4d44-aea9-159b0a1b00d3'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:13.578320 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-3ba0a83d-0c56-4d44-aea9-159b0a1b00d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e5b3edbfe2743a0854e53923d366cf2', 'request_id': 'req-3ba0a83d-0c56-4d44-aea9-159b0a1b00d3'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:13.588888 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:13.591787 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-25054ab5-6a79-4d28-91ef-91638a6c2544 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7f5f90833a94eb5999ebfcd8cd634f2'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:13.593383 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-3ba0a83d-0c56-4d44-aea9-159b0a1b00d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e5b3edbfe2743a0854e53923d366cf2', 'request_id': 'req-3ba0a83d-0c56-4d44-aea9-159b0a1b00d3'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:13.607803 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-25054ab5-6a79-4d28-91ef-91638a6c2544 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7f5f90833a94eb5999ebfcd8cd634f2'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:13.608767 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-25054ab5-6a79-4d28-91ef-91638a6c2544 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7f5f90833a94eb5999ebfcd8cd634f2'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:13.612739 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-3ba0a83d-0c56-4d44-aea9-159b0a1b00d3 admin admin] The token cache is being invalidate because project 4e5b3edbfe2743a0854e53923d366cf2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:13:13.613376 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-3ba0a83d-0c56-4d44-aea9-159b0a1b00d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:13.614331 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-3ba0a83d-0c56-4d44-aea9-159b0a1b00d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:13.616841 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 871/1758] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 9 12:13:13 2022] DELETE /identity/v3/projects/4e5b3edbfe2743a0854e53923d366cf2 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:13:13.617989 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba0a83d-0c56-4d44-aea9-159b0a1b00d3 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:13.618976 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 888/1759] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 9 12:13:13 2022] DELETE /identity/v3/users/e7f5f90833a94eb5999ebfcd8cd634f2 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:13:13.619590 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-3ba0a83d-0c56-4d44-aea9-159b0a1b00d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:13.623291 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baa3bb2-f8c5-4c97-bc64-a08d19368983 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:13.623568 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baa3bb2-f8c5-4c97-bc64-a08d19368983 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:13.624043 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baa3bb2-f8c5-4c97-bc64-a08d19368983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:13.662636 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-4baa3bb2-f8c5-4c97-bc64-a08d19368983 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:13:13.664640 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-4baa3bb2-f8c5-4c97-bc64-a08d19368983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:13.742528 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 889/1760] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:13:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:13:13.744843 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f67c09-3992-4156-aece-916138fa9c14 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:13.745471 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f67c09-3992-4156-aece-916138fa9c14 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:13.745991 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f67c09-3992-4156-aece-916138fa9c14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:13.754804 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba0a83d-0c56-4d44-aea9-159b0a1b00d3 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:13.769107 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-02f67c09-3992-4156-aece-916138fa9c14 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:13:13.770648 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-02f67c09-3992-4156-aece-916138fa9c14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:13.780709 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 890/1761] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:13:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:13:13.782500 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37b5433-8ccc-4426-bce6-6b9275c7a849 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:13.783007 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37b5433-8ccc-4426-bce6-6b9275c7a849 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:13.783485 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37b5433-8ccc-4426-bce6-6b9275c7a849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:13.806285 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-b37b5433-8ccc-4426-bce6-6b9275c7a849 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:13:13.807873 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-b37b5433-8ccc-4426-bce6-6b9275c7a849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:13.821860 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 891/1762] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:13:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:13:13.823275 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37b5433-8ccc-4426-bce6-6b9275c7a849 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:13.825034 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-b37b5433-8ccc-4426-bce6-6b9275c7a849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:13.861628 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317f90f2-fe97-421c-af15-7cdf2de63c24 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70bac169d5cd43e1926f310344ac11fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:13.862305 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317f90f2-fe97-421c-af15-7cdf2de63c24 service nova] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:13.862586 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317f90f2-fe97-421c-af15-7cdf2de63c24 service nova] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:13.863773 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317f90f2-fe97-421c-af15-7cdf2de63c24 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:13.866266 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-317f90f2-fe97-421c-af15-7cdf2de63c24 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:13.882499 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317f90f2-fe97-421c-af15-7cdf2de63c24 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:13.884453 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317f90f2-fe97-421c-af15-7cdf2de63c24 service nova] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:13.898561 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37b5433-8ccc-4426-bce6-6b9275c7a849 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:13.904624 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc88d390-4ad9-457b-9af3-c993ec2b6e49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:13.905346 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc88d390-4ad9-457b-9af3-c993ec2b6e49 service neutron] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:13.905698 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc88d390-4ad9-457b-9af3-c993ec2b6e49 service neutron] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:13.906274 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc88d390-4ad9-457b-9af3-c993ec2b6e49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:13.911216 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 872/1763] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:13:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:13:13.917745 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317f90f2-fe97-421c-af15-7cdf2de63c24 service nova] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:13.921044 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc88d390-4ad9-457b-9af3-c993ec2b6e49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:13.923197 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc88d390-4ad9-457b-9af3-c993ec2b6e49 service neutron] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:13.935227 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:13.944067 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 892/1764] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:13:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:13:13.949591 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317f90f2-fe97-421c-af15-7cdf2de63c24 service nova] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:13.955460 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c132316-7abb-4a76-81aa-47376d1e8300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:13.956265 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c132316-7abb-4a76-81aa-47376d1e8300 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:13.956969 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c132316-7abb-4a76-81aa-47376d1e8300 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:13.957346 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c132316-7abb-4a76-81aa-47376d1e8300 admin admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:13.958082 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c132316-7abb-4a76-81aa-47376d1e8300 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:13:13.958494 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c132316-7abb-4a76-81aa-47376d1e8300 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:13.961646 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c132316-7abb-4a76-81aa-47376d1e8300 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:13.978713 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 873/1765] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 9 12:13:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:13:13.982980 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc88d390-4ad9-457b-9af3-c993ec2b6e49 service neutron] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:13.997041 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc88d390-4ad9-457b-9af3-c993ec2b6e49 service neutron] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:14.007419 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce16428e-92d9-4998-bdee-261705e1e246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:14.008246 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce16428e-92d9-4998-bdee-261705e1e246 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:14.008524 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce16428e-92d9-4998-bdee-261705e1e246 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:14.008847 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce16428e-92d9-4998-bdee-261705e1e246 admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:14.009500 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce16428e-92d9-4998-bdee-261705e1e246 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:14.013143 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce16428e-92d9-4998-bdee-261705e1e246 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:14.085351 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 893/1766] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 9 12:13:13 2022] POST /identity/v3/projects => generated 383 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:13:14.090391 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c132316-7abb-4a76-81aa-47376d1e8300 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:14.104981 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c132316-7abb-4a76-81aa-47376d1e8300 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:14.302081 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a6cea2-f555-4232-85bc-afc9a61b42af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:14.302636 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a6cea2-f555-4232-85bc-afc9a61b42af admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:14.302952 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a6cea2-f555-4232-85bc-afc9a61b42af admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:14.303217 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a6cea2-f555-4232-85bc-afc9a61b42af admin admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:14.303856 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a6cea2-f555-4232-85bc-afc9a61b42af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:14.305806 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a6cea2-f555-4232-85bc-afc9a61b42af admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:14.358856 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 874/1767] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:13:14 2022] POST /identity/v3/users => generated 292 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:13:14.359707 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce16428e-92d9-4998-bdee-261705e1e246 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:14.373872 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce16428e-92d9-4998-bdee-261705e1e246 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:14.381100 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c9ce00-6e0e-4342-afca-b2b2e6f16764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:14.381955 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c9ce00-6e0e-4342-afca-b2b2e6f16764 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:14.382376 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c9ce00-6e0e-4342-afca-b2b2e6f16764 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:14.382785 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c9ce00-6e0e-4342-afca-b2b2e6f16764 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:14.383665 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c9ce00-6e0e-4342-afca-b2b2e6f16764 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:14.385860 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c9ce00-6e0e-4342-afca-b2b2e6f16764 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:14.401765 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 894/1768] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:13:14 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 12:13:14.405981 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a6cea2-f555-4232-85bc-afc9a61b42af admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:14.427780 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a6cea2-f555-4232-85bc-afc9a61b42af admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:14.459423 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a41f099-f79f-4b66-a47e-80dfdfaaf81d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:14.460052 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a41f099-f79f-4b66-a47e-80dfdfaaf81d admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:14.460328 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a41f099-f79f-4b66-a47e-80dfdfaaf81d admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:14.461310 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a41f099-f79f-4b66-a47e-80dfdfaaf81d admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:14.461976 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a41f099-f79f-4b66-a47e-80dfdfaaf81d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:14.465626 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a41f099-f79f-4b66-a47e-80dfdfaaf81d admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:14.504200 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 875/1769] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:13:14 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:13:14.509363 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c9ce00-6e0e-4342-afca-b2b2e6f16764 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:14.531164 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c9ce00-6e0e-4342-afca-b2b2e6f16764 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:14.547629 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f159ea6d-dc5a-45d5-b0de-740b0e3cf70e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:14.548326 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f159ea6d-dc5a-45d5-b0de-740b0e3cf70e admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:14.548608 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f159ea6d-dc5a-45d5-b0de-740b0e3cf70e admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:14.548889 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f159ea6d-dc5a-45d5-b0de-740b0e3cf70e admin admin] PATH_INFO: `/v3/projects/1f9fd7f0a81748af835121cc51d4172b/users/10c570150c5e4e079e6aaa468c66d281/roles/eecd2f160dd844f5906f3d345780c501` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:14.585125 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f159ea6d-dc5a-45d5-b0de-740b0e3cf70e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f9fd7f0a81748af835121cc51d4172b, user_id=10c570150c5e4e079e6aaa468c66d281, role_id=eecd2f160dd844f5906f3d345780c501)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:14.588545 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f159ea6d-dc5a-45d5-b0de-740b0e3cf70e admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:14.611715 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 895/1770] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:13:14 2022] PUT /identity/v3/projects/1f9fd7f0a81748af835121cc51d4172b/users/10c570150c5e4e079e6aaa468c66d281/roles/eecd2f160dd844f5906f3d345780c501 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:13:14.623648 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc97f569-1c9c-4627-8be3-67b5469938bd None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:14.623962 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc97f569-1c9c-4627-8be3-67b5469938bd None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:14.624206 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc97f569-1c9c-4627-8be3-67b5469938bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:14.687594 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-fc97f569-1c9c-4627-8be3-67b5469938bd None None] MFA Rules not processed for user `10c570150c5e4e079e6aaa468c66d281`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:13:14.688858 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-fc97f569-1c9c-4627-8be3-67b5469938bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:14.732904 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9b19f4-3092-4ce0-9b5a-8b1aa8cddafe None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:14.733254 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9b19f4-3092-4ce0-9b5a-8b1aa8cddafe None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:14.733432 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9b19f4-3092-4ce0-9b5a-8b1aa8cddafe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:14.776390 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-ef9b19f4-3092-4ce0-9b5a-8b1aa8cddafe None None] MFA Rules not processed for user `b9a5c9fb3f664ae097f0c3ce6f6c0764`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:13:14.778151 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-ef9b19f4-3092-4ce0-9b5a-8b1aa8cddafe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:14.816980 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:14.851773 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 876/1771] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:13:14 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:13:14.852761 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc97f569-1c9c-4627-8be3-67b5469938bd None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:14.854868 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-fc97f569-1c9c-4627-8be3-67b5469938bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:14.900506 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 896/1772] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:13:14 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:13:14.900810 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9b19f4-3092-4ce0-9b5a-8b1aa8cddafe None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:14.940436 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc97f569-1c9c-4627-8be3-67b5469938bd None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:14.945425 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f691e4e-767a-4e4c-8f75-55bf7fd58bdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:14.946430 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f691e4e-767a-4e4c-8f75-55bf7fd58bdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:14.946430 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f691e4e-767a-4e4c-8f75-55bf7fd58bdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:14.946697 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f691e4e-767a-4e4c-8f75-55bf7fd58bdb admin admin] PATH_INFO: `/v3/projects/11c6b0f334604ca2b1a97ef326faafb8` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:14.961427 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f691e4e-767a-4e4c-8f75-55bf7fd58bdb admin admin] RBAC: Authorizing `identity:delete_project(project_id=11c6b0f334604ca2b1a97ef326faafb8)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:14.966202 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f691e4e-767a-4e4c-8f75-55bf7fd58bdb admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:14.968014 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:14.993598 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9b19f4-3092-4ce0-9b5a-8b1aa8cddafe None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:14.999068 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a3ed5d-77f9-445e-bbe1-8460a6442f66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:14.999890 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a3ed5d-77f9-445e-bbe1-8460a6442f66 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:15.000180 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a3ed5d-77f9-445e-bbe1-8460a6442f66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:15.000450 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a3ed5d-77f9-445e-bbe1-8460a6442f66 admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:15.001089 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a3ed5d-77f9-445e-bbe1-8460a6442f66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:15.012132 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a3ed5d-77f9-445e-bbe1-8460a6442f66 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:15.050659 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-8f691e4e-767a-4e4c-8f75-55bf7fd58bdb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11c6b0f334604ca2b1a97ef326faafb8', 'request_id': 'req-8f691e4e-767a-4e4c-8f75-55bf7fd58bdb'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:15.059513 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 897/1773] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 9 12:13:14 2022] POST /identity/v3/projects => generated 381 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:13:15.059701 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-8f691e4e-767a-4e4c-8f75-55bf7fd58bdb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11c6b0f334604ca2b1a97ef326faafb8', 'request_id': 'req-8f691e4e-767a-4e4c-8f75-55bf7fd58bdb'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:15.064463 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a3ed5d-77f9-445e-bbe1-8460a6442f66 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:15.068930 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-8f691e4e-767a-4e4c-8f75-55bf7fd58bdb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11c6b0f334604ca2b1a97ef326faafb8', 'request_id': 'req-8f691e4e-767a-4e4c-8f75-55bf7fd58bdb'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:15.090561 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-8f691e4e-767a-4e4c-8f75-55bf7fd58bdb admin admin] The token cache is being invalidate because project 11c6b0f334604ca2b1a97ef326faafb8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:13:15.091196 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-8f691e4e-767a-4e4c-8f75-55bf7fd58bdb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:15.092390 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-8f691e4e-767a-4e4c-8f75-55bf7fd58bdb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:15.094832 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 877/1774] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 9 12:13:14 2022] DELETE /identity/v3/projects/11c6b0f334604ca2b1a97ef326faafb8 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:13:15.107846 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f691e4e-767a-4e4c-8f75-55bf7fd58bdb admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:15.109435 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-8f691e4e-767a-4e4c-8f75-55bf7fd58bdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:15.269889 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a3ed5d-77f9-445e-bbe1-8460a6442f66 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:15.292352 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670ddf59-26aa-4910-83be-837bb64b8f36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:15.292891 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670ddf59-26aa-4910-83be-837bb64b8f36 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:15.293172 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670ddf59-26aa-4910-83be-837bb64b8f36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:15.293448 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670ddf59-26aa-4910-83be-837bb64b8f36 admin admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:15.294092 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670ddf59-26aa-4910-83be-837bb64b8f36 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:15.295163 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f691e4e-767a-4e4c-8f75-55bf7fd58bdb admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:15.296132 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670ddf59-26aa-4910-83be-837bb64b8f36 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:15.302369 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49bc51b-6f3d-4243-af9e-7560379afb7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:15.303396 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49bc51b-6f3d-4243-af9e-7560379afb7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:15.303904 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49bc51b-6f3d-4243-af9e-7560379afb7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:15.304391 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49bc51b-6f3d-4243-af9e-7560379afb7c admin admin] PATH_INFO: `/v3/users/030e1b15a9d449ff8ed811c897c23350` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:15.306445 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49bc51b-6f3d-4243-af9e-7560379afb7c admin admin] RBAC: Authorizing `identity:delete_user(user_id=030e1b15a9d449ff8ed811c897c23350)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:15.309187 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49bc51b-6f3d-4243-af9e-7560379afb7c admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:15.370784 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 898/1775] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:13:15 2022] POST /identity/v3/users => generated 297 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:13:15.375834 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670ddf59-26aa-4910-83be-837bb64b8f36 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:15.378059 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-670ddf59-26aa-4910-83be-837bb64b8f36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:15.409353 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-a49bc51b-6f3d-4243-af9e-7560379afb7c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '030e1b15a9d449ff8ed811c897c23350'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:15.420819 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670ddf59-26aa-4910-83be-837bb64b8f36 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:15.426366 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c68c5e4-18cd-4e0c-964e-86d8e41fc13c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:15.427127 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c68c5e4-18cd-4e0c-964e-86d8e41fc13c admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:15.427408 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c68c5e4-18cd-4e0c-964e-86d8e41fc13c admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:15.427788 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c68c5e4-18cd-4e0c-964e-86d8e41fc13c admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:15.428787 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c68c5e4-18cd-4e0c-964e-86d8e41fc13c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:15.429303 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-a49bc51b-6f3d-4243-af9e-7560379afb7c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '030e1b15a9d449ff8ed811c897c23350'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:15.430415 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c68c5e4-18cd-4e0c-964e-86d8e41fc13c admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:15.440482 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-a49bc51b-6f3d-4243-af9e-7560379afb7c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '030e1b15a9d449ff8ed811c897c23350'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:15.447196 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 899/1776] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:13:15 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:13:15.449556 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c68c5e4-18cd-4e0c-964e-86d8e41fc13c admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:15.460442 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-a49bc51b-6f3d-4243-af9e-7560379afb7c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '030e1b15a9d449ff8ed811c897c23350'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:15.461486 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-a49bc51b-6f3d-4243-af9e-7560379afb7c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '030e1b15a9d449ff8ed811c897c23350'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:15.471147 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c68c5e4-18cd-4e0c-964e-86d8e41fc13c admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:15.480014 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 878/1777] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 9 12:13:15 2022] DELETE /identity/v3/users/030e1b15a9d449ff8ed811c897c23350 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:13:15.482795 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f25b73-b9a1-4ba0-8077-239f3b6d6947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:15.483901 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f25b73-b9a1-4ba0-8077-239f3b6d6947 admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:15.484448 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f25b73-b9a1-4ba0-8077-239f3b6d6947 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:15.485036 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f25b73-b9a1-4ba0-8077-239f3b6d6947 admin admin] PATH_INFO: `/v3/projects/017fa52e5be942faa8de984bd6af90a5/users/9f533fcaca604faab133e9991c5dc2fa/roles/3f2ca33902a04b6dbc2ef7d06d7289d3` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:15.559588 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f25b73-b9a1-4ba0-8077-239f3b6d6947 admin admin] RBAC: Authorizing `identity:create_grant(project_id=017fa52e5be942faa8de984bd6af90a5, user_id=9f533fcaca604faab133e9991c5dc2fa, role_id=3f2ca33902a04b6dbc2ef7d06d7289d3)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:15.577919 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f25b73-b9a1-4ba0-8077-239f3b6d6947 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:15.613940 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 900/1778] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:13:15 2022] PUT /identity/v3/projects/017fa52e5be942faa8de984bd6af90a5/users/9f533fcaca604faab133e9991c5dc2fa/roles/3f2ca33902a04b6dbc2ef7d06d7289d3 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:13:15.618304 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49bc51b-6f3d-4243-af9e-7560379afb7c admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:15.621227 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-a49bc51b-6f3d-4243-af9e-7560379afb7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:15.640053 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:15.787064 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49bc51b-6f3d-4243-af9e-7560379afb7c admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:15.857673 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2726dea4-b6a1-4c08-972a-7f7bdec1f9f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:15.858834 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2726dea4-b6a1-4c08-972a-7f7bdec1f9f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:15.859230 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2726dea4-b6a1-4c08-972a-7f7bdec1f9f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:15.859580 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2726dea4-b6a1-4c08-972a-7f7bdec1f9f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:15.860346 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2726dea4-b6a1-4c08-972a-7f7bdec1f9f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:15.862071 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2726dea4-b6a1-4c08-972a-7f7bdec1f9f3 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:15.874503 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 879/1779] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:13:15 2022] GET /identity/v3/roles => generated 1420 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:13:15.878554 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f25b73-b9a1-4ba0-8077-239f3b6d6947 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:15.914834 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f25b73-b9a1-4ba0-8077-239f3b6d6947 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:15.931470 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c82b15-85e9-40e0-8034-102f4b746e3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:15.933658 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c82b15-85e9-40e0-8034-102f4b746e3e admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:15.934035 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c82b15-85e9-40e0-8034-102f4b746e3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:15.935136 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c82b15-85e9-40e0-8034-102f4b746e3e admin admin] PATH_INFO: `/v3/projects/017fa52e5be942faa8de984bd6af90a5/users/9f533fcaca604faab133e9991c5dc2fa/roles/3f2ca33902a04b6dbc2ef7d06d7289d3` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:15.942858 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c82b15-85e9-40e0-8034-102f4b746e3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=017fa52e5be942faa8de984bd6af90a5, user_id=9f533fcaca604faab133e9991c5dc2fa, role_id=3f2ca33902a04b6dbc2ef7d06d7289d3)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:15.950512 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c82b15-85e9-40e0-8034-102f4b746e3e admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:15.953856 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:15.967193 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 901/1780] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:13:15 2022] PUT /identity/v3/projects/017fa52e5be942faa8de984bd6af90a5/users/9f533fcaca604faab133e9991c5dc2fa/roles/3f2ca33902a04b6dbc2ef7d06d7289d3 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:13:15.977551 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f65226-cd78-4e98-be79-5a45f1069cb1 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:15.977961 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f65226-cd78-4e98-be79-5a45f1069cb1 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:15.978962 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f65226-cd78-4e98-be79-5a45f1069cb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:16.030317 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-f5f65226-cd78-4e98-be79-5a45f1069cb1 None None] MFA Rules not processed for user `9f533fcaca604faab133e9991c5dc2fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:13:16.031750 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-f5f65226-cd78-4e98-be79-5a45f1069cb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:16.052540 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c82b15-85e9-40e0-8034-102f4b746e3e admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:16.059656 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-63c82b15-85e9-40e0-8034-102f4b746e3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:16.144578 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 880/1781] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:13:15 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:13:16.157698 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f65226-cd78-4e98-be79-5a45f1069cb1 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:16.159738 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-f5f65226-cd78-4e98-be79-5a45f1069cb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:16.216838 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:16.225544 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c82b15-85e9-40e0-8034-102f4b746e3e admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:16.232802 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125b890f-3d21-48f6-ae0c-f3b660d6d656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:16.233458 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125b890f-3d21-48f6-ae0c-f3b660d6d656 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:16.233736 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125b890f-3d21-48f6-ae0c-f3b660d6d656 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:16.234004 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125b890f-3d21-48f6-ae0c-f3b660d6d656 admin admin] PATH_INFO: `/v3/projects/81bc85bbb7bc41498fa58b25f0ab1309` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:16.235600 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125b890f-3d21-48f6-ae0c-f3b660d6d656 admin admin] RBAC: Authorizing `identity:delete_project(project_id=81bc85bbb7bc41498fa58b25f0ab1309)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:16.239452 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125b890f-3d21-48f6-ae0c-f3b660d6d656 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:16.240567 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f65226-cd78-4e98-be79-5a45f1069cb1 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:16.251750 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e8af9f-0344-422e-a63f-47910ac84220 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535c258100df40b58a5ee44b2c21d621', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:16.252400 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e8af9f-0344-422e-a63f-47910ac84220 service cinder] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:16.252674 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e8af9f-0344-422e-a63f-47910ac84220 service cinder] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:16.252944 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e8af9f-0344-422e-a63f-47910ac84220 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:16.284369 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e8af9f-0344-422e-a63f-47910ac84220 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:16.286685 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e8af9f-0344-422e-a63f-47910ac84220 service cinder] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:16.338849 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 881/1782] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:13:16 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:13:16.355680 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-125b890f-3d21-48f6-ae0c-f3b660d6d656 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81bc85bbb7bc41498fa58b25f0ab1309', 'request_id': 'req-125b890f-3d21-48f6-ae0c-f3b660d6d656'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:16.364660 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-125b890f-3d21-48f6-ae0c-f3b660d6d656 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81bc85bbb7bc41498fa58b25f0ab1309', 'request_id': 'req-125b890f-3d21-48f6-ae0c-f3b660d6d656'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:16.382845 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-125b890f-3d21-48f6-ae0c-f3b660d6d656 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81bc85bbb7bc41498fa58b25f0ab1309', 'request_id': 'req-125b890f-3d21-48f6-ae0c-f3b660d6d656'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:16.408252 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-125b890f-3d21-48f6-ae0c-f3b660d6d656 admin admin] The token cache is being invalidate because project 81bc85bbb7bc41498fa58b25f0ab1309 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81591) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:13:16.408703 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-125b890f-3d21-48f6-ae0c-f3b660d6d656 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:16.409414 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-125b890f-3d21-48f6-ae0c-f3b660d6d656 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:16.414028 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 902/1783] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 9 12:13:16 2022] DELETE /identity/v3/projects/81bc85bbb7bc41498fa58b25f0ab1309 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:13:16.438193 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7db3d3-c70d-48e8-a83b-b4d9fc843d9e None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:16.438507 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7db3d3-c70d-48e8-a83b-b4d9fc843d9e None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:16.438782 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7db3d3-c70d-48e8-a83b-b4d9fc843d9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:16.519766 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125b890f-3d21-48f6-ae0c-f3b660d6d656 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:16.521294 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-125b890f-3d21-48f6-ae0c-f3b660d6d656 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:16.526329 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-de7db3d3-c70d-48e8-a83b-b4d9fc843d9e None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:13:16.534246 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-de7db3d3-c70d-48e8-a83b-b4d9fc843d9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:16.646967 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125b890f-3d21-48f6-ae0c-f3b660d6d656 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:16.661017 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b99c29-c060-4c88-add2-2418f20859dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535c258100df40b58a5ee44b2c21d621', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:16.662170 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b99c29-c060-4c88-add2-2418f20859dd service cinder] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:16.662523 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b99c29-c060-4c88-add2-2418f20859dd service cinder] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:16.663285 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b99c29-c060-4c88-add2-2418f20859dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:16.666153 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-08b99c29-c060-4c88-add2-2418f20859dd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:16.676147 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 882/1784] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:13:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:13:16.688036 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29ac293-4958-44cb-8511-e038981f16ca None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:16.688326 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29ac293-4958-44cb-8511-e038981f16ca None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:16.688619 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29ac293-4958-44cb-8511-e038981f16ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:16.732240 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-f29ac293-4958-44cb-8511-e038981f16ca None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:13:16.733471 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-f29ac293-4958-44cb-8511-e038981f16ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:16.752287 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:16.758493 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 883/1785] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:13:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:13:16.764824 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29ac293-4958-44cb-8511-e038981f16ca None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:16.778414 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29ac293-4958-44cb-8511-e038981f16ca None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:16.784278 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be7c24b-eb82-4263-961d-8699f30d4cc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:16.784908 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be7c24b-eb82-4263-961d-8699f30d4cc4 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:16.785404 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be7c24b-eb82-4263-961d-8699f30d4cc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:16.785915 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be7c24b-eb82-4263-961d-8699f30d4cc4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:16.786490 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be7c24b-eb82-4263-961d-8699f30d4cc4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:13:16.786919 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be7c24b-eb82-4263-961d-8699f30d4cc4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:16.788769 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be7c24b-eb82-4263-961d-8699f30d4cc4 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:16.807763 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 884/1786] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 9 12:13:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:13:16.809928 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be7c24b-eb82-4263-961d-8699f30d4cc4 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:16.821585 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b99c29-c060-4c88-add2-2418f20859dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:16.825748 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b99c29-c060-4c88-add2-2418f20859dd service cinder] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:16.828580 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be7c24b-eb82-4263-961d-8699f30d4cc4 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:16.835814 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7eb1af-d0c7-4fa1-b9b0-ae935dc04d86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:16.839258 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7eb1af-d0c7-4fa1-b9b0-ae935dc04d86 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:16.839560 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7eb1af-d0c7-4fa1-b9b0-ae935dc04d86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:16.839890 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7eb1af-d0c7-4fa1-b9b0-ae935dc04d86 admin admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:16.841675 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7eb1af-d0c7-4fa1-b9b0-ae935dc04d86 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:16.847136 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7eb1af-d0c7-4fa1-b9b0-ae935dc04d86 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:16.848808 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 903/1787] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:13:16 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:13:16.923011 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 885/1788] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 9 12:13:16 2022] POST /identity/v3/projects => generated 383 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:13:16.927401 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b99c29-c060-4c88-add2-2418f20859dd service cinder] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:16.953780 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b99c29-c060-4c88-add2-2418f20859dd service cinder] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:17.106989 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93e3604-c6c7-4724-8581-89ba6d63d28a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:17.107964 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93e3604-c6c7-4724-8581-89ba6d63d28a admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:17.108432 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93e3604-c6c7-4724-8581-89ba6d63d28a admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:17.108805 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93e3604-c6c7-4724-8581-89ba6d63d28a admin admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:17.109537 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93e3604-c6c7-4724-8581-89ba6d63d28a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:17.111726 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93e3604-c6c7-4724-8581-89ba6d63d28a admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:17.141930 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 904/1789] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:13:16 2022] POST /identity/v3/users => generated 292 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:13:17.146680 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7eb1af-d0c7-4fa1-b9b0-ae935dc04d86 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:17.163274 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7eb1af-d0c7-4fa1-b9b0-ae935dc04d86 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:17.170652 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2361844-9013-4282-b3fd-edee0ed7a445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:17.171523 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2361844-9013-4282-b3fd-edee0ed7a445 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:17.171894 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2361844-9013-4282-b3fd-edee0ed7a445 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:17.172258 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2361844-9013-4282-b3fd-edee0ed7a445 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:17.173095 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2361844-9013-4282-b3fd-edee0ed7a445 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:17.175146 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2361844-9013-4282-b3fd-edee0ed7a445 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:17.191709 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 886/1790] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:13:17 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:13:17.192934 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93e3604-c6c7-4724-8581-89ba6d63d28a admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:17.212483 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93e3604-c6c7-4724-8581-89ba6d63d28a admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:17.220909 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d98756-a17b-4468-ada4-c03a4cc5e175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:17.221711 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d98756-a17b-4468-ada4-c03a4cc5e175 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:17.221974 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d98756-a17b-4468-ada4-c03a4cc5e175 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:17.222322 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d98756-a17b-4468-ada4-c03a4cc5e175 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:17.223117 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d98756-a17b-4468-ada4-c03a4cc5e175 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:17.224812 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d98756-a17b-4468-ada4-c03a4cc5e175 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:17.238615 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 905/1791] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:13:17 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 12:13:17.242856 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2361844-9013-4282-b3fd-edee0ed7a445 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:17.262304 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2361844-9013-4282-b3fd-edee0ed7a445 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:17.267830 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b094cd8-8910-47d2-9c9d-75f01e5573dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:17.268580 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b094cd8-8910-47d2-9c9d-75f01e5573dc admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:17.268968 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b094cd8-8910-47d2-9c9d-75f01e5573dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:17.269363 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b094cd8-8910-47d2-9c9d-75f01e5573dc admin admin] PATH_INFO: `/v3/projects/8c3ddb240fd247ce959270ea2514a047/users/d84edbc0091a4e75bdf79d7744dcbe15/roles/eecd2f160dd844f5906f3d345780c501` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:17.287313 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b094cd8-8910-47d2-9c9d-75f01e5573dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c3ddb240fd247ce959270ea2514a047, user_id=d84edbc0091a4e75bdf79d7744dcbe15, role_id=eecd2f160dd844f5906f3d345780c501)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:17.290098 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b094cd8-8910-47d2-9c9d-75f01e5573dc admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:17.308844 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 887/1792] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:13:17 2022] PUT /identity/v3/projects/8c3ddb240fd247ce959270ea2514a047/users/d84edbc0091a4e75bdf79d7744dcbe15/roles/eecd2f160dd844f5906f3d345780c501 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:13:17.320140 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11d5939-4a44-4da6-a7f5-269281435a91 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:17.320435 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11d5939-4a44-4da6-a7f5-269281435a91 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:17.320871 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11d5939-4a44-4da6-a7f5-269281435a91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:17.389446 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-b11d5939-4a44-4da6-a7f5-269281435a91 None None] MFA Rules not processed for user `d84edbc0091a4e75bdf79d7744dcbe15`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:13:17.395548 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-b11d5939-4a44-4da6-a7f5-269281435a91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:17.522974 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 906/1793] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:13:17 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:13:17.527882 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b094cd8-8910-47d2-9c9d-75f01e5573dc admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:17.658690 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b094cd8-8910-47d2-9c9d-75f01e5573dc admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:17.680485 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a2d734-862b-40ec-a84b-0d67d2bc1a80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:17.681328 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a2d734-862b-40ec-a84b-0d67d2bc1a80 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:17.681752 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a2d734-862b-40ec-a84b-0d67d2bc1a80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:17.682147 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a2d734-862b-40ec-a84b-0d67d2bc1a80 admin admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:17.682942 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a2d734-862b-40ec-a84b-0d67d2bc1a80 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:17.685129 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a2d734-862b-40ec-a84b-0d67d2bc1a80 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:17.706936 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:17.738945 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 888/1794] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 9 12:13:17 2022] POST /identity/v3/projects => generated 381 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:13:17.741152 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11d5939-4a44-4da6-a7f5-269281435a91 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:17.754498 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11d5939-4a44-4da6-a7f5-269281435a91 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:17.813073 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:17.827931 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ee034e-23f0-4678-ae9c-3be4c742655a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:17.828475 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ee034e-23f0-4678-ae9c-3be4c742655a admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:17.828754 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ee034e-23f0-4678-ae9c-3be4c742655a admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:17.829025 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ee034e-23f0-4678-ae9c-3be4c742655a admin admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:17.829661 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ee034e-23f0-4678-ae9c-3be4c742655a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:17.831651 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ee034e-23f0-4678-ae9c-3be4c742655a admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:17.853000 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 907/1795] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:13:17 2022] POST /identity/v3/users => generated 297 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:13:17.857196 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a2d734-862b-40ec-a84b-0d67d2bc1a80 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:17.867302 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a2d734-862b-40ec-a84b-0d67d2bc1a80 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:17.872480 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82379557-bb68-4293-9063-ae8776096f86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:17.873079 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82379557-bb68-4293-9063-ae8776096f86 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:17.873359 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82379557-bb68-4293-9063-ae8776096f86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:17.873634 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82379557-bb68-4293-9063-ae8776096f86 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:17.874275 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82379557-bb68-4293-9063-ae8776096f86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:17.875936 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82379557-bb68-4293-9063-ae8776096f86 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:17.886333 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 889/1796] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:13:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:13:17.890366 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ee034e-23f0-4678-ae9c-3be4c742655a admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:17.900406 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ee034e-23f0-4678-ae9c-3be4c742655a admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:17.905023 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af97813-c7b9-408f-8213-adcd679c763a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:17.905670 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af97813-c7b9-408f-8213-adcd679c763a admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:17.905943 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af97813-c7b9-408f-8213-adcd679c763a admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:17.906220 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af97813-c7b9-408f-8213-adcd679c763a admin admin] PATH_INFO: `/v3/projects/100179704bc84f02a538d7e7828a1cd5/users/f19a0124534546d288fdfc2090d7def5/roles/3f2ca33902a04b6dbc2ef7d06d7289d3` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:17.920578 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af97813-c7b9-408f-8213-adcd679c763a admin admin] RBAC: Authorizing `identity:create_grant(project_id=100179704bc84f02a538d7e7828a1cd5, user_id=f19a0124534546d288fdfc2090d7def5, role_id=3f2ca33902a04b6dbc2ef7d06d7289d3)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:17.923263 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af97813-c7b9-408f-8213-adcd679c763a admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:17.937088 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 908/1797] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:13:17 2022] PUT /identity/v3/projects/100179704bc84f02a538d7e7828a1cd5/users/f19a0124534546d288fdfc2090d7def5/roles/3f2ca33902a04b6dbc2ef7d06d7289d3 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:13:17.941017 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82379557-bb68-4293-9063-ae8776096f86 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:17.951456 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82379557-bb68-4293-9063-ae8776096f86 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:18.007957 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dad137-8e6b-45c7-b194-cf58060392fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:18.008576 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dad137-8e6b-45c7-b194-cf58060392fa admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:18.008855 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dad137-8e6b-45c7-b194-cf58060392fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:18.009130 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dad137-8e6b-45c7-b194-cf58060392fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:18.010420 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dad137-8e6b-45c7-b194-cf58060392fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:18.012123 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dad137-8e6b-45c7-b194-cf58060392fa admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:18.022058 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 890/1798] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:13:17 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:13:18.026303 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af97813-c7b9-408f-8213-adcd679c763a admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:18.036461 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af97813-c7b9-408f-8213-adcd679c763a admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:18.041058 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e52c1ec-ae2b-4e3c-a0d3-cfc117617ac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:18.041742 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e52c1ec-ae2b-4e3c-a0d3-cfc117617ac7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:18.042023 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e52c1ec-ae2b-4e3c-a0d3-cfc117617ac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:18.042296 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e52c1ec-ae2b-4e3c-a0d3-cfc117617ac7 admin admin] PATH_INFO: `/v3/projects/100179704bc84f02a538d7e7828a1cd5/users/f19a0124534546d288fdfc2090d7def5/roles/3f2ca33902a04b6dbc2ef7d06d7289d3` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:18.044274 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e52c1ec-ae2b-4e3c-a0d3-cfc117617ac7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=100179704bc84f02a538d7e7828a1cd5, user_id=f19a0124534546d288fdfc2090d7def5, role_id=3f2ca33902a04b6dbc2ef7d06d7289d3)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:18.046886 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e52c1ec-ae2b-4e3c-a0d3-cfc117617ac7 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:18.057837 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 909/1799] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:13:18 2022] PUT /identity/v3/projects/100179704bc84f02a538d7e7828a1cd5/users/f19a0124534546d288fdfc2090d7def5/roles/3f2ca33902a04b6dbc2ef7d06d7289d3 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:13:18.068025 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f5694c-5bb9-478b-8f16-0f48fdf2294b None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:18.068312 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f5694c-5bb9-478b-8f16-0f48fdf2294b None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:18.068588 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f5694c-5bb9-478b-8f16-0f48fdf2294b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:18.097603 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-37f5694c-5bb9-478b-8f16-0f48fdf2294b None None] MFA Rules not processed for user `f19a0124534546d288fdfc2090d7def5`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:13:18.098816 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-37f5694c-5bb9-478b-8f16-0f48fdf2294b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:18.193039 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 891/1800] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:13:18 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:13:18.204737 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e52c1ec-ae2b-4e3c-a0d3-cfc117617ac7 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:18.206432 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-7e52c1ec-ae2b-4e3c-a0d3-cfc117617ac7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:18.346716 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e52c1ec-ae2b-4e3c-a0d3-cfc117617ac7 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:18.352076 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63779b3e-040a-4bec-a95b-9119753bf734 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535c258100df40b58a5ee44b2c21d621', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:18.352952 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63779b3e-040a-4bec-a95b-9119753bf734 service cinder] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:18.353961 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63779b3e-040a-4bec-a95b-9119753bf734 service cinder] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:18.354455 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63779b3e-040a-4bec-a95b-9119753bf734 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:18.378458 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63779b3e-040a-4bec-a95b-9119753bf734 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:18.384270 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63779b3e-040a-4bec-a95b-9119753bf734 service cinder] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:18.395789 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 910/1801] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:13:18 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:13:18.568167 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f5694c-5bb9-478b-8f16-0f48fdf2294b None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:18.578351 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f5694c-5bb9-478b-8f16-0f48fdf2294b None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:18.581943 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545d43c2-ba20-4a6f-9919-0671adf41382 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535c258100df40b58a5ee44b2c21d621', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:18.582558 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545d43c2-ba20-4a6f-9919-0671adf41382 service cinder] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:18.582841 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545d43c2-ba20-4a6f-9919-0671adf41382 service cinder] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:18.583148 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545d43c2-ba20-4a6f-9919-0671adf41382 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:18.671176 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545d43c2-ba20-4a6f-9919-0671adf41382 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:18.672700 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545d43c2-ba20-4a6f-9919-0671adf41382 service cinder] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:18.677152 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:18.685062 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 892/1802] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:13:18 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:13:19.266999 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63779b3e-040a-4bec-a95b-9119753bf734 service cinder] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:19.373587 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63779b3e-040a-4bec-a95b-9119753bf734 service cinder] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:19.378287 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9f3976-ed03-43e4-a69a-342083645a76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:19.379216 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9f3976-ed03-43e4-a69a-342083645a76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:19.379731 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9f3976-ed03-43e4-a69a-342083645a76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:19.381024 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9f3976-ed03-43e4-a69a-342083645a76 admin admin] PATH_INFO: `/v3/users/d84edbc0091a4e75bdf79d7744dcbe15` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:19.382389 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9f3976-ed03-43e4-a69a-342083645a76 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d84edbc0091a4e75bdf79d7744dcbe15)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:19.385006 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9f3976-ed03-43e4-a69a-342083645a76 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:19.440556 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=89,keystone:INSERT=5,keystone:DELETE=6 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:19.445231 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-bc9f3976-ed03-43e4-a69a-342083645a76 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd84edbc0091a4e75bdf79d7744dcbe15'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:19.456289 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-bc9f3976-ed03-43e4-a69a-342083645a76 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd84edbc0091a4e75bdf79d7744dcbe15'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:19.469094 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-bc9f3976-ed03-43e4-a69a-342083645a76 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd84edbc0091a4e75bdf79d7744dcbe15'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:19.489056 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-bc9f3976-ed03-43e4-a69a-342083645a76 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd84edbc0091a4e75bdf79d7744dcbe15'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:19.489707 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-bc9f3976-ed03-43e4-a69a-342083645a76 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd84edbc0091a4e75bdf79d7744dcbe15'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:19.497886 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 911/1803] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 9 12:13:19 2022] DELETE /identity/v3/users/d84edbc0091a4e75bdf79d7744dcbe15 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:13:19.502504 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea31635d-34f8-4d90-818d-246ae6720334 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:19.503026 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea31635d-34f8-4d90-818d-246ae6720334 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:19.503202 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea31635d-34f8-4d90-818d-246ae6720334 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:19.528098 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-ea31635d-34f8-4d90-818d-246ae6720334 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:13:19.529350 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-ea31635d-34f8-4d90-818d-246ae6720334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:19.602947 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 893/1804] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:13:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:13:19.613150 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9f3976-ed03-43e4-a69a-342083645a76 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:19.616365 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-bc9f3976-ed03-43e4-a69a-342083645a76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:19.709659 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9f3976-ed03-43e4-a69a-342083645a76 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:19.709827 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea31635d-34f8-4d90-818d-246ae6720334 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:19.711201 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-ea31635d-34f8-4d90-818d-246ae6720334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:19.714460 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efcde42-3838-41f0-83b2-b94d2dd7ac3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:19.715210 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efcde42-3838-41f0-83b2-b94d2dd7ac3a service neutron] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:19.715583 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efcde42-3838-41f0-83b2-b94d2dd7ac3a service neutron] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:19.715947 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efcde42-3838-41f0-83b2-b94d2dd7ac3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:19.724235 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea31635d-34f8-4d90-818d-246ae6720334 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:19.726614 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efcde42-3838-41f0-83b2-b94d2dd7ac3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:19.730140 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efcde42-3838-41f0-83b2-b94d2dd7ac3a service neutron] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:19.738115 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daec5893-c1ae-4323-9ae1-0ee7fca3b7b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:19.739083 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daec5893-c1ae-4323-9ae1-0ee7fca3b7b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:19.739480 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daec5893-c1ae-4323-9ae1-0ee7fca3b7b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:19.739711 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daec5893-c1ae-4323-9ae1-0ee7fca3b7b2 admin admin] PATH_INFO: `/v3/users/c07d3a751567467ea4fd94334f1a96ca` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:19.740940 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daec5893-c1ae-4323-9ae1-0ee7fca3b7b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c07d3a751567467ea4fd94334f1a96ca)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:19.741374 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 912/1805] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:13:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:13:19.750980 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daec5893-c1ae-4323-9ae1-0ee7fca3b7b2 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:19.804443 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-daec5893-c1ae-4323-9ae1-0ee7fca3b7b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c07d3a751567467ea4fd94334f1a96ca'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:19.818841 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-daec5893-c1ae-4323-9ae1-0ee7fca3b7b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c07d3a751567467ea4fd94334f1a96ca'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:19.827704 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-daec5893-c1ae-4323-9ae1-0ee7fca3b7b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c07d3a751567467ea4fd94334f1a96ca'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:19.835377 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-daec5893-c1ae-4323-9ae1-0ee7fca3b7b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c07d3a751567467ea4fd94334f1a96ca'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:19.836342 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-daec5893-c1ae-4323-9ae1-0ee7fca3b7b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c07d3a751567467ea4fd94334f1a96ca'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:19.843247 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 894/1806] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 9 12:13:19 2022] DELETE /identity/v3/users/c07d3a751567467ea4fd94334f1a96ca => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:13:19.847467 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a359da3-285d-452b-a843-85a5c8c78672 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:19.847753 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a359da3-285d-452b-a843-85a5c8c78672 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:19.848185 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a359da3-285d-452b-a843-85a5c8c78672 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:19.868579 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-1a359da3-285d-452b-a843-85a5c8c78672 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:13:19.870507 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-1a359da3-285d-452b-a843-85a5c8c78672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:19.969554 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 913/1807] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:13:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:13:19.983141 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daec5893-c1ae-4323-9ae1-0ee7fca3b7b2 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:19.986254 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-daec5893-c1ae-4323-9ae1-0ee7fca3b7b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:20.062788 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daec5893-c1ae-4323-9ae1-0ee7fca3b7b2 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:20.066222 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05c12ba-c9bc-4bdb-b3dc-ee1253556ea4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:20.067123 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05c12ba-c9bc-4bdb-b3dc-ee1253556ea4 service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:20.067537 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05c12ba-c9bc-4bdb-b3dc-ee1253556ea4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:20.067816 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05c12ba-c9bc-4bdb-b3dc-ee1253556ea4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:20.082724 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05c12ba-c9bc-4bdb-b3dc-ee1253556ea4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:20.084288 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05c12ba-c9bc-4bdb-b3dc-ee1253556ea4 service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:20.096173 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 895/1808] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:13:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:13:20.669849 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a359da3-285d-452b-a843-85a5c8c78672 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:20.672193 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-1a359da3-285d-452b-a843-85a5c8c78672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:20.695021 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a359da3-285d-452b-a843-85a5c8c78672 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:20.704128 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad524bf0-4733-4593-baf1-f36dcde0624f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:20.705136 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad524bf0-4733-4593-baf1-f36dcde0624f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:20.705656 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad524bf0-4733-4593-baf1-f36dcde0624f admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:20.706306 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad524bf0-4733-4593-baf1-f36dcde0624f admin admin] PATH_INFO: `/v3/projects/8c3ddb240fd247ce959270ea2514a047` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:20.708906 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad524bf0-4733-4593-baf1-f36dcde0624f admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c3ddb240fd247ce959270ea2514a047)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:20.711418 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad524bf0-4733-4593-baf1-f36dcde0624f admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:20.811280 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-ad524bf0-4733-4593-baf1-f36dcde0624f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c3ddb240fd247ce959270ea2514a047', 'request_id': 'req-ad524bf0-4733-4593-baf1-f36dcde0624f'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:20.820041 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-ad524bf0-4733-4593-baf1-f36dcde0624f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c3ddb240fd247ce959270ea2514a047', 'request_id': 'req-ad524bf0-4733-4593-baf1-f36dcde0624f'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:20.821443 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:20.830050 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-ad524bf0-4733-4593-baf1-f36dcde0624f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c3ddb240fd247ce959270ea2514a047', 'request_id': 'req-ad524bf0-4733-4593-baf1-f36dcde0624f'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:20.864625 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-ad524bf0-4733-4593-baf1-f36dcde0624f admin admin] The token cache is being invalidate because project 8c3ddb240fd247ce959270ea2514a047 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81591) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:13:20.865353 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-ad524bf0-4733-4593-baf1-f36dcde0624f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:20.866355 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-ad524bf0-4733-4593-baf1-f36dcde0624f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:20.873676 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 914/1809] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 9 12:13:20 2022] DELETE /identity/v3/projects/8c3ddb240fd247ce959270ea2514a047 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:13:20.877893 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05c12ba-c9bc-4bdb-b3dc-ee1253556ea4 service neutron] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:20.898015 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad524bf0-4733-4593-baf1-f36dcde0624f admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:20.904289 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05c12ba-c9bc-4bdb-b3dc-ee1253556ea4 service neutron] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:20.906104 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-ad524bf0-4733-4593-baf1-f36dcde0624f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:20.928176 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:21.009981 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad524bf0-4733-4593-baf1-f36dcde0624f admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:21.018125 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3c480c-e1fd-4d70-8d34-a2cb878ce176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:21.018842 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3c480c-e1fd-4d70-8d34-a2cb878ce176 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:21.019837 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3c480c-e1fd-4d70-8d34-a2cb878ce176 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:21.019837 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3c480c-e1fd-4d70-8d34-a2cb878ce176 admin admin] PATH_INFO: `/v3/users/10c570150c5e4e079e6aaa468c66d281` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:21.021754 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3c480c-e1fd-4d70-8d34-a2cb878ce176 admin admin] RBAC: Authorizing `identity:delete_user(user_id=10c570150c5e4e079e6aaa468c66d281)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:21.025608 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3c480c-e1fd-4d70-8d34-a2cb878ce176 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:21.037991 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53680fd-dabc-409e-9c9a-94b5e001234f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:21.038847 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53680fd-dabc-409e-9c9a-94b5e001234f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:21.040332 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53680fd-dabc-409e-9c9a-94b5e001234f admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:21.040332 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53680fd-dabc-409e-9c9a-94b5e001234f admin admin] PATH_INFO: `/v3/users/f19a0124534546d288fdfc2090d7def5` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:21.043978 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53680fd-dabc-409e-9c9a-94b5e001234f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f19a0124534546d288fdfc2090d7def5)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:21.046039 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53680fd-dabc-409e-9c9a-94b5e001234f admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:21.155260 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-a53680fd-dabc-409e-9c9a-94b5e001234f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f19a0124534546d288fdfc2090d7def5'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:21.182433 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-cc3c480c-e1fd-4d70-8d34-a2cb878ce176 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10c570150c5e4e079e6aaa468c66d281'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:21.198432 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-cc3c480c-e1fd-4d70-8d34-a2cb878ce176 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10c570150c5e4e079e6aaa468c66d281'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:21.213040 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-a53680fd-dabc-409e-9c9a-94b5e001234f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f19a0124534546d288fdfc2090d7def5'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:21.221156 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-cc3c480c-e1fd-4d70-8d34-a2cb878ce176 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10c570150c5e4e079e6aaa468c66d281'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:21.230348 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-a53680fd-dabc-409e-9c9a-94b5e001234f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f19a0124534546d288fdfc2090d7def5'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:21.245217 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-cc3c480c-e1fd-4d70-8d34-a2cb878ce176 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10c570150c5e4e079e6aaa468c66d281'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:21.246463 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-cc3c480c-e1fd-4d70-8d34-a2cb878ce176 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10c570150c5e4e079e6aaa468c66d281'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:21.257633 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 915/1810] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 9 12:13:20 2022] DELETE /identity/v3/users/10c570150c5e4e079e6aaa468c66d281 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:13:21.262950 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-a53680fd-dabc-409e-9c9a-94b5e001234f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f19a0124534546d288fdfc2090d7def5'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:21.264003 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-a53680fd-dabc-409e-9c9a-94b5e001234f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f19a0124534546d288fdfc2090d7def5'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:21.268776 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2711ed-26b0-420d-8837-ea6f0fec7ebf None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:21.269563 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2711ed-26b0-420d-8837-ea6f0fec7ebf None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:21.270067 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2711ed-26b0-420d-8837-ea6f0fec7ebf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:21.273848 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 896/1811] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 9 12:13:20 2022] DELETE /identity/v3/users/f19a0124534546d288fdfc2090d7def5 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:13:21.303661 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-5d2711ed-26b0-420d-8837-ea6f0fec7ebf None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:13:21.305132 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-5d2711ed-26b0-420d-8837-ea6f0fec7ebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:21.383152 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 916/1812] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:13:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:13:21.428835 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53680fd-dabc-409e-9c9a-94b5e001234f admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:21.430852 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-a53680fd-dabc-409e-9c9a-94b5e001234f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:21.528114 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53680fd-dabc-409e-9c9a-94b5e001234f admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:21.532170 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916bad18-56dd-4e68-a091-16e6c8d43bae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:21.532821 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916bad18-56dd-4e68-a091-16e6c8d43bae service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:21.533101 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916bad18-56dd-4e68-a091-16e6c8d43bae service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:21.533371 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916bad18-56dd-4e68-a091-16e6c8d43bae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:21.546622 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916bad18-56dd-4e68-a091-16e6c8d43bae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:21.550824 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916bad18-56dd-4e68-a091-16e6c8d43bae service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:21.566155 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 897/1813] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:13:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:13:21.725613 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2711ed-26b0-420d-8837-ea6f0fec7ebf None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:21.728042 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-5d2711ed-26b0-420d-8837-ea6f0fec7ebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:21.761996 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2711ed-26b0-420d-8837-ea6f0fec7ebf None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:21.769481 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6c7eae-ad2f-4900-a03c-63741cbb1747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:21.770398 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6c7eae-ad2f-4900-a03c-63741cbb1747 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:21.771217 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6c7eae-ad2f-4900-a03c-63741cbb1747 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:21.772153 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6c7eae-ad2f-4900-a03c-63741cbb1747 admin admin] PATH_INFO: `/v3/projects/d9613b08ac104a45b9796d38a180839a` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:21.773834 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6c7eae-ad2f-4900-a03c-63741cbb1747 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9613b08ac104a45b9796d38a180839a)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:21.776656 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6c7eae-ad2f-4900-a03c-63741cbb1747 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:21.778333 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:21.870085 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-5d6c7eae-ad2f-4900-a03c-63741cbb1747 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9613b08ac104a45b9796d38a180839a', 'request_id': 'req-5d6c7eae-ad2f-4900-a03c-63741cbb1747'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:21.877860 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-5d6c7eae-ad2f-4900-a03c-63741cbb1747 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9613b08ac104a45b9796d38a180839a', 'request_id': 'req-5d6c7eae-ad2f-4900-a03c-63741cbb1747'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:21.888866 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-5d6c7eae-ad2f-4900-a03c-63741cbb1747 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9613b08ac104a45b9796d38a180839a', 'request_id': 'req-5d6c7eae-ad2f-4900-a03c-63741cbb1747'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:21.912660 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-5d6c7eae-ad2f-4900-a03c-63741cbb1747 admin admin] The token cache is being invalidate because project d9613b08ac104a45b9796d38a180839a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81591) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:13:21.913175 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-5d6c7eae-ad2f-4900-a03c-63741cbb1747 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:21.915941 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-5d6c7eae-ad2f-4900-a03c-63741cbb1747 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:21.918301 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 917/1814] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 9 12:13:21 2022] DELETE /identity/v3/projects/d9613b08ac104a45b9796d38a180839a => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:13:21.922802 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916bad18-56dd-4e68-a091-16e6c8d43bae service neutron] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:21.942294 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916bad18-56dd-4e68-a091-16e6c8d43bae service neutron] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:22.010458 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:22.022532 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6c7eae-ad2f-4900-a03c-63741cbb1747 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:22.024526 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-5d6c7eae-ad2f-4900-a03c-63741cbb1747 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:22.087474 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41adfc79-bf05-4504-a6a7-43c6a11cae6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:22.089122 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41adfc79-bf05-4504-a6a7-43c6a11cae6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:22.091735 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41adfc79-bf05-4504-a6a7-43c6a11cae6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:22.092142 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41adfc79-bf05-4504-a6a7-43c6a11cae6c admin admin] PATH_INFO: `/v3/users/1a936453f7734fb092d27deb0a3b1569` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:22.094504 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41adfc79-bf05-4504-a6a7-43c6a11cae6c admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a936453f7734fb092d27deb0a3b1569)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:22.099415 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41adfc79-bf05-4504-a6a7-43c6a11cae6c admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:22.174067 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6c7eae-ad2f-4900-a03c-63741cbb1747 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:22.191823 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a5ff73-78ae-4a72-b539-b2eb82317495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:22.192992 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a5ff73-78ae-4a72-b539-b2eb82317495 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:22.194196 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a5ff73-78ae-4a72-b539-b2eb82317495 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:22.194488 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a5ff73-78ae-4a72-b539-b2eb82317495 admin admin] PATH_INFO: `/v3/projects/100179704bc84f02a538d7e7828a1cd5` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:22.196484 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a5ff73-78ae-4a72-b539-b2eb82317495 admin admin] RBAC: Authorizing `identity:delete_project(project_id=100179704bc84f02a538d7e7828a1cd5)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:22.203910 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a5ff73-78ae-4a72-b539-b2eb82317495 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:22.207315 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-41adfc79-bf05-4504-a6a7-43c6a11cae6c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a936453f7734fb092d27deb0a3b1569'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:22.243434 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-41adfc79-bf05-4504-a6a7-43c6a11cae6c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a936453f7734fb092d27deb0a3b1569'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:22.262931 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-41adfc79-bf05-4504-a6a7-43c6a11cae6c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a936453f7734fb092d27deb0a3b1569'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:22.286734 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-41adfc79-bf05-4504-a6a7-43c6a11cae6c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a936453f7734fb092d27deb0a3b1569'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:22.287965 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-41adfc79-bf05-4504-a6a7-43c6a11cae6c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a936453f7734fb092d27deb0a3b1569'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:22.305265 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 898/1815] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 9 12:13:21 2022] DELETE /identity/v3/users/1a936453f7734fb092d27deb0a3b1569 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:13:22.309442 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41adfc79-bf05-4504-a6a7-43c6a11cae6c admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:22.340251 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-41adfc79-bf05-4504-a6a7-43c6a11cae6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:22.360020 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-f4a5ff73-78ae-4a72-b539-b2eb82317495 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '100179704bc84f02a538d7e7828a1cd5', 'request_id': 'req-f4a5ff73-78ae-4a72-b539-b2eb82317495'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:22.370416 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-f4a5ff73-78ae-4a72-b539-b2eb82317495 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '100179704bc84f02a538d7e7828a1cd5', 'request_id': 'req-f4a5ff73-78ae-4a72-b539-b2eb82317495'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:22.380455 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-f4a5ff73-78ae-4a72-b539-b2eb82317495 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '100179704bc84f02a538d7e7828a1cd5', 'request_id': 'req-f4a5ff73-78ae-4a72-b539-b2eb82317495'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:22.403026 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-f4a5ff73-78ae-4a72-b539-b2eb82317495 admin admin] The token cache is being invalidate because project 100179704bc84f02a538d7e7828a1cd5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81591) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:13:22.403555 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-f4a5ff73-78ae-4a72-b539-b2eb82317495 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:22.404318 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-f4a5ff73-78ae-4a72-b539-b2eb82317495 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:22.406932 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 918/1816] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 9 12:13:22 2022] DELETE /identity/v3/projects/100179704bc84f02a538d7e7828a1cd5 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:13:22.413306 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78a8b84-10c9-41bf-b5ac-c3a2394dad38 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:22.413741 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78a8b84-10c9-41bf-b5ac-c3a2394dad38 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:22.414020 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78a8b84-10c9-41bf-b5ac-c3a2394dad38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:22.440423 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-d78a8b84-10c9-41bf-b5ac-c3a2394dad38 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:13:22.441959 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-d78a8b84-10c9-41bf-b5ac-c3a2394dad38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:22.445428 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41adfc79-bf05-4504-a6a7-43c6a11cae6c admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:22.460821 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:22.518018 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae116cfe-7af8-4f2f-86f8-1934036e4798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:22.518801 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae116cfe-7af8-4f2f-86f8-1934036e4798 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:22.519107 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae116cfe-7af8-4f2f-86f8-1934036e4798 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:22.519375 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae116cfe-7af8-4f2f-86f8-1934036e4798 admin admin] PATH_INFO: `/v3/projects/1f9fd7f0a81748af835121cc51d4172b` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:22.520560 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae116cfe-7af8-4f2f-86f8-1934036e4798 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f9fd7f0a81748af835121cc51d4172b)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:22.522927 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae116cfe-7af8-4f2f-86f8-1934036e4798 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:22.526464 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:22.528270 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 919/1817] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:13:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:13:22.537889 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f5bb07-0ed4-4861-a36e-3ba41d80fb08 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:22.538167 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f5bb07-0ed4-4861-a36e-3ba41d80fb08 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:22.538622 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f5bb07-0ed4-4861-a36e-3ba41d80fb08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:22.582283 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-f2f5bb07-0ed4-4861-a36e-3ba41d80fb08 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:13:22.583781 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-f2f5bb07-0ed4-4861-a36e-3ba41d80fb08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:22.584128 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-ae116cfe-7af8-4f2f-86f8-1934036e4798 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f9fd7f0a81748af835121cc51d4172b', 'request_id': 'req-ae116cfe-7af8-4f2f-86f8-1934036e4798'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:22.597940 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-ae116cfe-7af8-4f2f-86f8-1934036e4798 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f9fd7f0a81748af835121cc51d4172b', 'request_id': 'req-ae116cfe-7af8-4f2f-86f8-1934036e4798'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:22.608669 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-ae116cfe-7af8-4f2f-86f8-1934036e4798 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f9fd7f0a81748af835121cc51d4172b', 'request_id': 'req-ae116cfe-7af8-4f2f-86f8-1934036e4798'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:22.623955 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-ae116cfe-7af8-4f2f-86f8-1934036e4798 admin admin] The token cache is being invalidate because project 1f9fd7f0a81748af835121cc51d4172b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:13:22.624297 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-ae116cfe-7af8-4f2f-86f8-1934036e4798 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:22.624973 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-ae116cfe-7af8-4f2f-86f8-1934036e4798 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:22.626718 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 899/1818] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 9 12:13:22 2022] DELETE /identity/v3/projects/1f9fd7f0a81748af835121cc51d4172b => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:13:22.630719 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae116cfe-7af8-4f2f-86f8-1934036e4798 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:22.632502 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-ae116cfe-7af8-4f2f-86f8-1934036e4798 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:22.736162 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae116cfe-7af8-4f2f-86f8-1934036e4798 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:22.737327 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 920/1819] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:13:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:13:22.741570 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f5bb07-0ed4-4861-a36e-3ba41d80fb08 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:22.745944 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f733b6b-0b0e-481f-8a19-f5b6325da0cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:22.747291 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f733b6b-0b0e-481f-8a19-f5b6325da0cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:22.747676 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f733b6b-0b0e-481f-8a19-f5b6325da0cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:22.748715 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f733b6b-0b0e-481f-8a19-f5b6325da0cc admin admin] PATH_INFO: `/v3/users/9f533fcaca604faab133e9991c5dc2fa` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:22.751024 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f733b6b-0b0e-481f-8a19-f5b6325da0cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f533fcaca604faab133e9991c5dc2fa)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:22.752331 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f5bb07-0ed4-4861-a36e-3ba41d80fb08 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:22.753315 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f733b6b-0b0e-481f-8a19-f5b6325da0cc admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:22.774129 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278eb1de-4e96-4a6d-97e1-4d20e588ecba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:22.775463 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278eb1de-4e96-4a6d-97e1-4d20e588ecba admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:22.776002 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278eb1de-4e96-4a6d-97e1-4d20e588ecba admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:22.776573 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278eb1de-4e96-4a6d-97e1-4d20e588ecba admin admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:22.777488 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278eb1de-4e96-4a6d-97e1-4d20e588ecba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:13:22.777945 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278eb1de-4e96-4a6d-97e1-4d20e588ecba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:22.781273 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278eb1de-4e96-4a6d-97e1-4d20e588ecba admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:22.803114 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 921/1820] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 9 12:13:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:13:22.814724 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278eb1de-4e96-4a6d-97e1-4d20e588ecba admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:22.831343 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-7f733b6b-0b0e-481f-8a19-f5b6325da0cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f533fcaca604faab133e9991c5dc2fa'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:22.839445 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278eb1de-4e96-4a6d-97e1-4d20e588ecba admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:22.849709 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-7f733b6b-0b0e-481f-8a19-f5b6325da0cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f533fcaca604faab133e9991c5dc2fa'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:22.851824 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d9de19-8d5f-498c-a810-525cad4bf15c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:22.852454 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d9de19-8d5f-498c-a810-525cad4bf15c admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:22.852736 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d9de19-8d5f-498c-a810-525cad4bf15c admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:22.853598 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d9de19-8d5f-498c-a810-525cad4bf15c admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:22.854644 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d9de19-8d5f-498c-a810-525cad4bf15c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:22.856629 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d9de19-8d5f-498c-a810-525cad4bf15c admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:22.862799 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-7f733b6b-0b0e-481f-8a19-f5b6325da0cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f533fcaca604faab133e9991c5dc2fa'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:22.875131 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-7f733b6b-0b0e-481f-8a19-f5b6325da0cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f533fcaca604faab133e9991c5dc2fa'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:22.876882 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-7f733b6b-0b0e-481f-8a19-f5b6325da0cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f533fcaca604faab133e9991c5dc2fa'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:22.888348 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 900/1821] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 9 12:13:22 2022] DELETE /identity/v3/users/9f533fcaca604faab133e9991c5dc2fa => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:13:22.893233 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 922/1822] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 9 12:13:22 2022] POST /identity/v3/projects => generated 375 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:13:22.897937 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f733b6b-0b0e-481f-8a19-f5b6325da0cc admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:22.899601 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-7f733b6b-0b0e-481f-8a19-f5b6325da0cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:22.914505 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:22.980074 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f733b6b-0b0e-481f-8a19-f5b6325da0cc admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:23.051793 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9773d8-7993-4ae3-a1e4-856a1c768643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:23.052694 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9773d8-7993-4ae3-a1e4-856a1c768643 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:23.053039 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9773d8-7993-4ae3-a1e4-856a1c768643 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:23.053381 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9773d8-7993-4ae3-a1e4-856a1c768643 admin admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:23.054164 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9773d8-7993-4ae3-a1e4-856a1c768643 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:23.057213 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9773d8-7993-4ae3-a1e4-856a1c768643 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:23.087145 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 901/1823] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:13:22 2022] POST /identity/v3/users => generated 288 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:13:23.090938 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d9de19-8d5f-498c-a810-525cad4bf15c admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:23.105866 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d9de19-8d5f-498c-a810-525cad4bf15c admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:23.122096 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34ca9e8-b79c-4b87-8e0e-a9b3ace455d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:23.122728 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34ca9e8-b79c-4b87-8e0e-a9b3ace455d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:23.123041 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34ca9e8-b79c-4b87-8e0e-a9b3ace455d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:23.123318 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34ca9e8-b79c-4b87-8e0e-a9b3ace455d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:23.123971 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34ca9e8-b79c-4b87-8e0e-a9b3ace455d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:23.125575 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34ca9e8-b79c-4b87-8e0e-a9b3ace455d3 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:23.144362 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 923/1824] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:13:23 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:13:23.146059 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9773d8-7993-4ae3-a1e4-856a1c768643 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:23.173476 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9773d8-7993-4ae3-a1e4-856a1c768643 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:23.189245 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f45967-b887-4f70-9ef1-4db0c827c667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:23.189886 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f45967-b887-4f70-9ef1-4db0c827c667 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:23.191546 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f45967-b887-4f70-9ef1-4db0c827c667 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:23.191884 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f45967-b887-4f70-9ef1-4db0c827c667 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:23.193050 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f45967-b887-4f70-9ef1-4db0c827c667 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:23.194309 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f45967-b887-4f70-9ef1-4db0c827c667 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:23.217128 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 902/1825] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:13:23 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:13:23.219333 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34ca9e8-b79c-4b87-8e0e-a9b3ace455d3 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:23.236298 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34ca9e8-b79c-4b87-8e0e-a9b3ace455d3 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:23.248630 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70676ff6-cd11-4b5c-a5b2-9ab75770e92e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:23.250515 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70676ff6-cd11-4b5c-a5b2-9ab75770e92e admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:23.250973 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70676ff6-cd11-4b5c-a5b2-9ab75770e92e admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:23.251728 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70676ff6-cd11-4b5c-a5b2-9ab75770e92e admin admin] PATH_INFO: `/v3/projects/8aa271ab4bd94e6f8269f67601936ad9/users/9993833b9e3648b6b4f256a3eb6e358d/roles/eecd2f160dd844f5906f3d345780c501` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:23.281344 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70676ff6-cd11-4b5c-a5b2-9ab75770e92e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8aa271ab4bd94e6f8269f67601936ad9, user_id=9993833b9e3648b6b4f256a3eb6e358d, role_id=eecd2f160dd844f5906f3d345780c501)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:23.284744 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70676ff6-cd11-4b5c-a5b2-9ab75770e92e admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:23.313172 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 924/1826] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:13:23 2022] PUT /identity/v3/projects/8aa271ab4bd94e6f8269f67601936ad9/users/9993833b9e3648b6b4f256a3eb6e358d/roles/eecd2f160dd844f5906f3d345780c501 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:13:23.315253 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f45967-b887-4f70-9ef1-4db0c827c667 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:23.316195 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-26f45967-b887-4f70-9ef1-4db0c827c667 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:23.322671 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5af5000-7ea8-461e-8fac-42f70d1e79ef None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:23.323662 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5af5000-7ea8-461e-8fac-42f70d1e79ef None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:23.324043 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5af5000-7ea8-461e-8fac-42f70d1e79ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:23.336786 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f45967-b887-4f70-9ef1-4db0c827c667 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:23.369756 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-d5af5000-7ea8-461e-8fac-42f70d1e79ef None None] MFA Rules not processed for user `9993833b9e3648b6b4f256a3eb6e358d`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:13:23.371677 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-d5af5000-7ea8-461e-8fac-42f70d1e79ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:23.458170 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:23.501351 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 925/1827] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:13:23 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:13:23.507218 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5af5000-7ea8-461e-8fac-42f70d1e79ef None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:23.511507 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d3ecdb-7fb8-4120-b691-f221ae714ee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:23.512169 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d3ecdb-7fb8-4120-b691-f221ae714ee6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:23.512457 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d3ecdb-7fb8-4120-b691-f221ae714ee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:23.512721 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d3ecdb-7fb8-4120-b691-f221ae714ee6 admin admin] PATH_INFO: `/v3/projects/cc342aa9bae54145bcdec06443c71380` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:23.513900 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d3ecdb-7fb8-4120-b691-f221ae714ee6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc342aa9bae54145bcdec06443c71380)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:23.517639 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d3ecdb-7fb8-4120-b691-f221ae714ee6 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:23.522289 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5af5000-7ea8-461e-8fac-42f70d1e79ef None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:23.533785 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817201fc-c5ec-4791-b001-71decc6f9b9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:23.534570 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817201fc-c5ec-4791-b001-71decc6f9b9b admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:23.534988 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817201fc-c5ec-4791-b001-71decc6f9b9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:23.535392 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817201fc-c5ec-4791-b001-71decc6f9b9b admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:23.537081 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817201fc-c5ec-4791-b001-71decc6f9b9b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:23.539690 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817201fc-c5ec-4791-b001-71decc6f9b9b admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:23.605622 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 926/1828] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 9 12:13:23 2022] POST /identity/v3/projects => generated 375 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:13:23.607214 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817201fc-c5ec-4791-b001-71decc6f9b9b admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:23.622878 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:23.639197 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-70d3ecdb-7fb8-4120-b691-f221ae714ee6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc342aa9bae54145bcdec06443c71380', 'request_id': 'req-70d3ecdb-7fb8-4120-b691-f221ae714ee6'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:23.654221 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-70d3ecdb-7fb8-4120-b691-f221ae714ee6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc342aa9bae54145bcdec06443c71380', 'request_id': 'req-70d3ecdb-7fb8-4120-b691-f221ae714ee6'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:23.671036 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-70d3ecdb-7fb8-4120-b691-f221ae714ee6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc342aa9bae54145bcdec06443c71380', 'request_id': 'req-70d3ecdb-7fb8-4120-b691-f221ae714ee6'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:23.694254 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-70d3ecdb-7fb8-4120-b691-f221ae714ee6 admin admin] The token cache is being invalidate because project cc342aa9bae54145bcdec06443c71380 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:13:23.694891 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-70d3ecdb-7fb8-4120-b691-f221ae714ee6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:23.696098 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-70d3ecdb-7fb8-4120-b691-f221ae714ee6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:23.699832 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 903/1829] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 9 12:13:23 2022] DELETE /identity/v3/projects/cc342aa9bae54145bcdec06443c71380 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:13:23.709098 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc1ad7f-4315-4d12-8d92-c251563b4e8c None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:23.710941 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc1ad7f-4315-4d12-8d92-c251563b4e8c None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:23.711761 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc1ad7f-4315-4d12-8d92-c251563b4e8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:23.761078 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-8cc1ad7f-4315-4d12-8d92-c251563b4e8c None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:13:23.762814 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-8cc1ad7f-4315-4d12-8d92-c251563b4e8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:23.780048 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817201fc-c5ec-4791-b001-71decc6f9b9b admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:23.866856 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a35c6d-d836-4931-905c-75d9eb3b7b32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:23.867635 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a35c6d-d836-4931-905c-75d9eb3b7b32 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:23.868012 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a35c6d-d836-4931-905c-75d9eb3b7b32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:23.868415 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a35c6d-d836-4931-905c-75d9eb3b7b32 admin admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:23.870700 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a35c6d-d836-4931-905c-75d9eb3b7b32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:23.873686 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a35c6d-d836-4931-905c-75d9eb3b7b32 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:23.909757 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 927/1830] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:13:23 2022] POST /identity/v3/users => generated 294 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:13:23.911599 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a35c6d-d836-4931-905c-75d9eb3b7b32 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:23.913903 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-f0a35c6d-d836-4931-905c-75d9eb3b7b32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:23.919545 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 904/1831] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:13:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:13:23.927190 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d9f4d6-25e2-45d7-b49c-4bd6a65de647 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:23.927473 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d9f4d6-25e2-45d7-b49c-4bd6a65de647 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:23.927744 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d9f4d6-25e2-45d7-b49c-4bd6a65de647 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:23.937496 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a35c6d-d836-4931-905c-75d9eb3b7b32 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:23.960008 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392d14a5-20df-461a-99f7-95332757af42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:23.960809 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392d14a5-20df-461a-99f7-95332757af42 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:23.961418 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392d14a5-20df-461a-99f7-95332757af42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:23.961814 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392d14a5-20df-461a-99f7-95332757af42 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:23.963529 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392d14a5-20df-461a-99f7-95332757af42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:23.965616 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392d14a5-20df-461a-99f7-95332757af42 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:23.979951 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-13d9f4d6-25e2-45d7-b49c-4bd6a65de647 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:13:23.985083 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-13d9f4d6-25e2-45d7-b49c-4bd6a65de647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:23.999648 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 928/1832] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:13:23 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:13:24.001226 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392d14a5-20df-461a-99f7-95332757af42 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:24.023559 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 905/1833] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:13:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:13:24.028913 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d9f4d6-25e2-45d7-b49c-4bd6a65de647 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:24.044015 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d9f4d6-25e2-45d7-b49c-4bd6a65de647 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:24.048832 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392d14a5-20df-461a-99f7-95332757af42 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:24.050037 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bacfff3-503f-4da4-a0ef-576369808448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:24.050916 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bacfff3-503f-4da4-a0ef-576369808448 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:24.051308 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bacfff3-503f-4da4-a0ef-576369808448 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:24.051749 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bacfff3-503f-4da4-a0ef-576369808448 admin admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:24.052754 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bacfff3-503f-4da4-a0ef-576369808448 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:13:24.053375 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bacfff3-503f-4da4-a0ef-576369808448 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:24.056627 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bacfff3-503f-4da4-a0ef-576369808448 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:24.072747 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63535f49-95e0-4797-9d24-726a623c49a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:24.073442 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63535f49-95e0-4797-9d24-726a623c49a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:24.073720 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63535f49-95e0-4797-9d24-726a623c49a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:24.073994 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63535f49-95e0-4797-9d24-726a623c49a7 admin admin] PATH_INFO: `/v3/projects/bb91af272330489790ddcae6cc82f2ab/users/89a9e2a6b5004f5fbac737b7c538fb69/roles/3f2ca33902a04b6dbc2ef7d06d7289d3` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:24.111938 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 906/1834] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 9 12:13:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:13:24.114505 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bacfff3-503f-4da4-a0ef-576369808448 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:24.140471 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63535f49-95e0-4797-9d24-726a623c49a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb91af272330489790ddcae6cc82f2ab, user_id=89a9e2a6b5004f5fbac737b7c538fb69, role_id=3f2ca33902a04b6dbc2ef7d06d7289d3)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:24.143207 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63535f49-95e0-4797-9d24-726a623c49a7 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:24.166457 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bacfff3-503f-4da4-a0ef-576369808448 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:24.195729 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cf881b-3efb-4aad-b200-928fd764f100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:24.200461 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 929/1835] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:13:23 2022] PUT /identity/v3/projects/bb91af272330489790ddcae6cc82f2ab/users/89a9e2a6b5004f5fbac737b7c538fb69/roles/3f2ca33902a04b6dbc2ef7d06d7289d3 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:13:24.201449 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cf881b-3efb-4aad-b200-928fd764f100 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:24.201586 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63535f49-95e0-4797-9d24-726a623c49a7 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:24.202797 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cf881b-3efb-4aad-b200-928fd764f100 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:24.204436 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cf881b-3efb-4aad-b200-928fd764f100 admin admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:24.205674 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cf881b-3efb-4aad-b200-928fd764f100 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:24.210323 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cf881b-3efb-4aad-b200-928fd764f100 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:24.258107 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:24.275517 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 907/1836] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 9 12:13:24 2022] POST /identity/v3/projects => generated 375 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:13:24.279514 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cf881b-3efb-4aad-b200-928fd764f100 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:24.453101 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63535f49-95e0-4797-9d24-726a623c49a7 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:24.534280 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cf881b-3efb-4aad-b200-928fd764f100 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:24.542233 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92efe5c-1b99-427b-88b3-39d5796225bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:24.543273 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92efe5c-1b99-427b-88b3-39d5796225bc admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:24.543624 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92efe5c-1b99-427b-88b3-39d5796225bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:24.544036 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92efe5c-1b99-427b-88b3-39d5796225bc admin admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:24.545288 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92efe5c-1b99-427b-88b3-39d5796225bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:24.547476 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92efe5c-1b99-427b-88b3-39d5796225bc admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:24.584442 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 908/1837] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:13:24 2022] POST /identity/v3/users => generated 288 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:13:24.586057 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92efe5c-1b99-427b-88b3-39d5796225bc admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:24.588519 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf994a5-3a33-4fc4-8ca1-d6b5557e7e80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:24.590990 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf994a5-3a33-4fc4-8ca1-d6b5557e7e80 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:24.591108 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf994a5-3a33-4fc4-8ca1-d6b5557e7e80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:24.591360 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf994a5-3a33-4fc4-8ca1-d6b5557e7e80 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:24.592040 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf994a5-3a33-4fc4-8ca1-d6b5557e7e80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:24.595796 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf994a5-3a33-4fc4-8ca1-d6b5557e7e80 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:24.605805 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92efe5c-1b99-427b-88b3-39d5796225bc admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:24.612101 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 930/1838] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:13:24 2022] GET /identity/v3/roles => generated 1420 bytes in 408 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 12:13:24.613490 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf994a5-3a33-4fc4-8ca1-d6b5557e7e80 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:24.616783 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35d178f-e0df-4377-a1eb-6903cfd666b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:24.617495 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35d178f-e0df-4377-a1eb-6903cfd666b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:24.617495 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35d178f-e0df-4377-a1eb-6903cfd666b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:24.620128 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35d178f-e0df-4377-a1eb-6903cfd666b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:24.622930 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35d178f-e0df-4377-a1eb-6903cfd666b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:24.633187 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35d178f-e0df-4377-a1eb-6903cfd666b1 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:24.640230 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf994a5-3a33-4fc4-8ca1-d6b5557e7e80 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:24.647233 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be806c72-0d99-4a01-a90a-641aae7314ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:24.648784 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be806c72-0d99-4a01-a90a-641aae7314ec admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:24.649214 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be806c72-0d99-4a01-a90a-641aae7314ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:24.649630 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be806c72-0d99-4a01-a90a-641aae7314ec admin admin] PATH_INFO: `/v3/projects/bb91af272330489790ddcae6cc82f2ab/users/89a9e2a6b5004f5fbac737b7c538fb69/roles/3f2ca33902a04b6dbc2ef7d06d7289d3` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:24.662226 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 909/1839] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:13:24 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:13:24.662226 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35d178f-e0df-4377-a1eb-6903cfd666b1 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:24.663815 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be806c72-0d99-4a01-a90a-641aae7314ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb91af272330489790ddcae6cc82f2ab, user_id=89a9e2a6b5004f5fbac737b7c538fb69, role_id=3f2ca33902a04b6dbc2ef7d06d7289d3)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:24.671651 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be806c72-0d99-4a01-a90a-641aae7314ec admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:24.674579 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:24.687062 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35d178f-e0df-4377-a1eb-6903cfd666b1 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:24.691401 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 931/1840] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:13:24 2022] PUT /identity/v3/projects/bb91af272330489790ddcae6cc82f2ab/users/89a9e2a6b5004f5fbac737b7c538fb69/roles/3f2ca33902a04b6dbc2ef7d06d7289d3 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:13:24.701499 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84ca4d5-aaf2-44ef-a8c1-273b370b28d4 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:24.702685 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84ca4d5-aaf2-44ef-a8c1-273b370b28d4 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:24.703091 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84ca4d5-aaf2-44ef-a8c1-273b370b28d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:24.741177 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-e84ca4d5-aaf2-44ef-a8c1-273b370b28d4 None None] MFA Rules not processed for user `89a9e2a6b5004f5fbac737b7c538fb69`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:13:24.742552 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-e84ca4d5-aaf2-44ef-a8c1-273b370b28d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:24.829372 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e2d04e-7da0-46bf-ab15-7c39627d6082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:24.830510 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e2d04e-7da0-46bf-ab15-7c39627d6082 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:24.831037 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e2d04e-7da0-46bf-ab15-7c39627d6082 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:24.831534 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e2d04e-7da0-46bf-ab15-7c39627d6082 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:24.832579 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e2d04e-7da0-46bf-ab15-7c39627d6082 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:24.834453 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e2d04e-7da0-46bf-ab15-7c39627d6082 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:24.852871 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 910/1841] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:13:24 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:13:24.855130 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e2d04e-7da0-46bf-ab15-7c39627d6082 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:24.861932 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 932/1842] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:13:24 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:13:24.867058 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e2d04e-7da0-46bf-ab15-7c39627d6082 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:24.870201 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84ca4d5-aaf2-44ef-a8c1-273b370b28d4 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:24.872601 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-e84ca4d5-aaf2-44ef-a8c1-273b370b28d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:24.872951 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7161340-4ce6-4873-b37e-eb86ae497c38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:24.874079 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7161340-4ce6-4873-b37e-eb86ae497c38 admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:24.874381 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7161340-4ce6-4873-b37e-eb86ae497c38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:24.874743 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7161340-4ce6-4873-b37e-eb86ae497c38 admin admin] PATH_INFO: `/v3/projects/f316f1990c07480fa7258787b1c2c47f/users/3503d4622e66454e8c81d718072d1a60/roles/eecd2f160dd844f5906f3d345780c501` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:24.895470 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7161340-4ce6-4873-b37e-eb86ae497c38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f316f1990c07480fa7258787b1c2c47f, user_id=3503d4622e66454e8c81d718072d1a60, role_id=eecd2f160dd844f5906f3d345780c501)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:24.898799 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7161340-4ce6-4873-b37e-eb86ae497c38 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:24.930508 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 911/1843] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:13:24 2022] PUT /identity/v3/projects/f316f1990c07480fa7258787b1c2c47f/users/3503d4622e66454e8c81d718072d1a60/roles/eecd2f160dd844f5906f3d345780c501 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:13:24.938582 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0154d1e-18fd-40e8-acb0-6b805f4ad961 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:24.939033 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0154d1e-18fd-40e8-acb0-6b805f4ad961 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:24.940985 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0154d1e-18fd-40e8-acb0-6b805f4ad961 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:24.942320 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:24.968555 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84ca4d5-aaf2-44ef-a8c1-273b370b28d4 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:24.984956 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-b0154d1e-18fd-40e8-acb0-6b805f4ad961 None None] MFA Rules not processed for user `3503d4622e66454e8c81d718072d1a60`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:13:24.986182 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-b0154d1e-18fd-40e8-acb0-6b805f4ad961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:25.041924 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f8a9c5-3a0e-450b-9520-d1182691b928 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535c258100df40b58a5ee44b2c21d621', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:25.042797 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f8a9c5-3a0e-450b-9520-d1182691b928 service cinder] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:25.043116 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f8a9c5-3a0e-450b-9520-d1182691b928 service cinder] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:25.043381 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f8a9c5-3a0e-450b-9520-d1182691b928 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:25.046892 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-62f8a9c5-3a0e-450b-9520-d1182691b928 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:25.072972 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 912/1844] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:13:24 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:13:25.074079 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0154d1e-18fd-40e8-acb0-6b805f4ad961 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:25.075570 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-b0154d1e-18fd-40e8-acb0-6b805f4ad961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:25.130850 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f8a9c5-3a0e-450b-9520-d1182691b928 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:25.132923 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f8a9c5-3a0e-450b-9520-d1182691b928 service cinder] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:25.151029 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 933/1845] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:13:24 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:13:25.152494 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f8a9c5-3a0e-450b-9520-d1182691b928 service cinder] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:25.154254 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-62f8a9c5-3a0e-450b-9520-d1182691b928 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:25.169244 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f8a9c5-3a0e-450b-9520-d1182691b928 service cinder] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:25.171673 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0154d1e-18fd-40e8-acb0-6b805f4ad961 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:25.177092 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffad85d4-02db-41b0-a0f6-c343f1499050 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535c258100df40b58a5ee44b2c21d621', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:25.178105 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffad85d4-02db-41b0-a0f6-c343f1499050 service cinder] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:25.179378 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffad85d4-02db-41b0-a0f6-c343f1499050 service cinder] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:25.180073 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffad85d4-02db-41b0-a0f6-c343f1499050 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:25.189899 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffad85d4-02db-41b0-a0f6-c343f1499050 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:25.191528 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffad85d4-02db-41b0-a0f6-c343f1499050 service cinder] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:25.193093 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b3d665-0299-4423-aa2e-0b72c73de020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:25.193909 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b3d665-0299-4423-aa2e-0b72c73de020 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:25.195023 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b3d665-0299-4423-aa2e-0b72c73de020 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:25.195826 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b3d665-0299-4423-aa2e-0b72c73de020 admin admin] PATH_INFO: `/v3/projects/017fa52e5be942faa8de984bd6af90a5` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:25.198415 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b3d665-0299-4423-aa2e-0b72c73de020 admin admin] RBAC: Authorizing `identity:delete_project(project_id=017fa52e5be942faa8de984bd6af90a5)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:25.200686 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b3d665-0299-4423-aa2e-0b72c73de020 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:25.203048 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 934/1846] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:13:25 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:13:25.211346 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:25.291015 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-17b3d665-0299-4423-aa2e-0b72c73de020 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '017fa52e5be942faa8de984bd6af90a5', 'request_id': 'req-17b3d665-0299-4423-aa2e-0b72c73de020'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:25.299494 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-17b3d665-0299-4423-aa2e-0b72c73de020 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '017fa52e5be942faa8de984bd6af90a5', 'request_id': 'req-17b3d665-0299-4423-aa2e-0b72c73de020'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:25.315539 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-17b3d665-0299-4423-aa2e-0b72c73de020 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '017fa52e5be942faa8de984bd6af90a5', 'request_id': 'req-17b3d665-0299-4423-aa2e-0b72c73de020'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:25.332675 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-17b3d665-0299-4423-aa2e-0b72c73de020 admin admin] The token cache is being invalidate because project 017fa52e5be942faa8de984bd6af90a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:13:25.333039 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-17b3d665-0299-4423-aa2e-0b72c73de020 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:25.333958 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-17b3d665-0299-4423-aa2e-0b72c73de020 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:25.336243 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 913/1847] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 9 12:13:25 2022] DELETE /identity/v3/projects/017fa52e5be942faa8de984bd6af90a5 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:13:25.342084 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec1bafd-830b-4cc0-8ef8-44d530602052 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:25.342871 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec1bafd-830b-4cc0-8ef8-44d530602052 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:25.343353 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec1bafd-830b-4cc0-8ef8-44d530602052 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:25.398875 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-1ec1bafd-830b-4cc0-8ef8-44d530602052 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:13:25.407859 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-1ec1bafd-830b-4cc0-8ef8-44d530602052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:25.590969 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 935/1848] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:13:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:13:25.595446 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a2c0f0-fbf3-4285-adaa-38fb999b1fec None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:25.595854 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a2c0f0-fbf3-4285-adaa-38fb999b1fec None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:25.596183 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a2c0f0-fbf3-4285-adaa-38fb999b1fec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:25.609855 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4e49ad-ced0-4415-bffa-53152cd112a5 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:25.610142 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4e49ad-ced0-4415-bffa-53152cd112a5 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:25.610478 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4e49ad-ced0-4415-bffa-53152cd112a5 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:25.612550 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 936/1849] 10.222.0.49 () {54 vars in 921 bytes} [Fri Sep 9 12:13:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 12:13:25.624170 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1806f5d1-9572-4452-bf1c-9a96bd7671da None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:25.624530 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1806f5d1-9572-4452-bf1c-9a96bd7671da None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:25.624832 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1806f5d1-9572-4452-bf1c-9a96bd7671da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:25.659993 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-1806f5d1-9572-4452-bf1c-9a96bd7671da None None] MFA Rules not processed for user `70bac169d5cd43e1926f310344ac11fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:13:25.661769 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-33a2c0f0-fbf3-4285-adaa-38fb999b1fec None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:13:25.662714 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-1806f5d1-9572-4452-bf1c-9a96bd7671da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:25.663025 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-33a2c0f0-fbf3-4285-adaa-38fb999b1fec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:25.697832 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 914/1850] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:13:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:13:25.698978 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a2c0f0-fbf3-4285-adaa-38fb999b1fec None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:25.728077 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a2c0f0-fbf3-4285-adaa-38fb999b1fec None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:25.750070 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:25.750472 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94090257-0e8d-4bff-9bc1-36c93bcc473c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:25.751806 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94090257-0e8d-4bff-9bc1-36c93bcc473c admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:25.752270 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94090257-0e8d-4bff-9bc1-36c93bcc473c admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:25.752641 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94090257-0e8d-4bff-9bc1-36c93bcc473c admin admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:25.754323 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94090257-0e8d-4bff-9bc1-36c93bcc473c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:13:25.754738 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94090257-0e8d-4bff-9bc1-36c93bcc473c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:25.771301 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94090257-0e8d-4bff-9bc1-36c93bcc473c admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:25.832234 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 915/1851] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 9 12:13:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:13:25.832970 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94090257-0e8d-4bff-9bc1-36c93bcc473c admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:25.880640 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 937/1852] 10.222.0.49 () {58 vars in 1018 bytes} [Fri Sep 9 12:13:25 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:13:25.885696 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94090257-0e8d-4bff-9bc1-36c93bcc473c admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:25.918127 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa1b477-a67e-4b38-97af-4b174e628505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:25.918970 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa1b477-a67e-4b38-97af-4b174e628505 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:25.919384 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa1b477-a67e-4b38-97af-4b174e628505 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:25.919880 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa1b477-a67e-4b38-97af-4b174e628505 admin admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:25.923242 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa1b477-a67e-4b38-97af-4b174e628505 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:25.928651 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa1b477-a67e-4b38-97af-4b174e628505 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:26.005075 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 916/1853] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 9 12:13:25 2022] POST /identity/v3/projects => generated 377 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:13:26.010859 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1806f5d1-9572-4452-bf1c-9a96bd7671da None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:26.031557 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1806f5d1-9572-4452-bf1c-9a96bd7671da None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:26.158544 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c87545-74b5-419a-a4f6-e5e58bdccebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:26.159704 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c87545-74b5-419a-a4f6-e5e58bdccebd admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:26.159704 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c87545-74b5-419a-a4f6-e5e58bdccebd admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:26.159880 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c87545-74b5-419a-a4f6-e5e58bdccebd admin admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:26.160384 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c87545-74b5-419a-a4f6-e5e58bdccebd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:26.162608 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c87545-74b5-419a-a4f6-e5e58bdccebd admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:26.223095 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 938/1854] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:13:26 2022] POST /identity/v3/users => generated 289 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:13:26.224608 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa1b477-a67e-4b38-97af-4b174e628505 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:26.237725 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c87545-74b5-419a-a4f6-e5e58bdccebd admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:26.241549 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa1b477-a67e-4b38-97af-4b174e628505 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:26.243141 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-b3c87545-74b5-419a-a4f6-e5e58bdccebd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:26.272319 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65123566-5dff-4867-975c-930256e03ec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:26.273227 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65123566-5dff-4867-975c-930256e03ec6 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:26.273505 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65123566-5dff-4867-975c-930256e03ec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:26.273771 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65123566-5dff-4867-975c-930256e03ec6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:26.274420 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65123566-5dff-4867-975c-930256e03ec6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:26.278734 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65123566-5dff-4867-975c-930256e03ec6 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:26.302127 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 917/1855] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:13:26 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:13:26.315199 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65123566-5dff-4867-975c-930256e03ec6 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:26.340560 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65123566-5dff-4867-975c-930256e03ec6 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:26.352596 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a494c2b2-3051-4a89-995a-209cfd681718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:26.353984 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a494c2b2-3051-4a89-995a-209cfd681718 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:26.354430 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a494c2b2-3051-4a89-995a-209cfd681718 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:26.354703 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a494c2b2-3051-4a89-995a-209cfd681718 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:26.355386 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a494c2b2-3051-4a89-995a-209cfd681718 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:26.356990 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a494c2b2-3051-4a89-995a-209cfd681718 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:26.378362 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 918/1856] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:13:26 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:13:26.381961 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a494c2b2-3051-4a89-995a-209cfd681718 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:26.399124 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a494c2b2-3051-4a89-995a-209cfd681718 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:26.405784 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382e6134-3c7e-4df9-a221-db4fbc06c0a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:26.406211 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382e6134-3c7e-4df9-a221-db4fbc06c0a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:26.406425 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382e6134-3c7e-4df9-a221-db4fbc06c0a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:26.406703 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382e6134-3c7e-4df9-a221-db4fbc06c0a8 admin admin] PATH_INFO: `/v3/projects/254937a622844299b5f0238075e81b1c/users/c5b39da495d64c5dbea3aea312f31586/roles/eecd2f160dd844f5906f3d345780c501` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:26.427880 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382e6134-3c7e-4df9-a221-db4fbc06c0a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=254937a622844299b5f0238075e81b1c, user_id=c5b39da495d64c5dbea3aea312f31586, role_id=eecd2f160dd844f5906f3d345780c501)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:26.432073 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382e6134-3c7e-4df9-a221-db4fbc06c0a8 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:26.439371 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c87545-74b5-419a-a4f6-e5e58bdccebd admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:26.455177 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 919/1857] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:13:26 2022] PUT /identity/v3/projects/254937a622844299b5f0238075e81b1c/users/c5b39da495d64c5dbea3aea312f31586/roles/eecd2f160dd844f5906f3d345780c501 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:13:26.465852 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494ca446-48ea-4c98-a1a3-2672f23bc3cc None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:26.466129 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494ca446-48ea-4c98-a1a3-2672f23bc3cc None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:26.466395 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494ca446-48ea-4c98-a1a3-2672f23bc3cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:26.498552 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-494ca446-48ea-4c98-a1a3-2672f23bc3cc None None] MFA Rules not processed for user `c5b39da495d64c5dbea3aea312f31586`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:13:26.499700 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-494ca446-48ea-4c98-a1a3-2672f23bc3cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:26.515226 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:26.539854 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a7099c-a7ad-4fa5-89bf-3f572aaa6af2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70bac169d5cd43e1926f310344ac11fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:26.541119 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a7099c-a7ad-4fa5-89bf-3f572aaa6af2 service nova] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:26.541614 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a7099c-a7ad-4fa5-89bf-3f572aaa6af2 service nova] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:26.541952 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a7099c-a7ad-4fa5-89bf-3f572aaa6af2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:26.560310 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a7099c-a7ad-4fa5-89bf-3f572aaa6af2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:26.562313 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a7099c-a7ad-4fa5-89bf-3f572aaa6af2 service nova] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:26.576408 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 939/1858] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:13:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:13:26.584168 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:26.594953 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 920/1859] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:13:26 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:13:26.597035 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a7099c-a7ad-4fa5-89bf-3f572aaa6af2 service nova] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:26.670274 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a7099c-a7ad-4fa5-89bf-3f572aaa6af2 service nova] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:26.676057 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2966039-7092-49d7-bb9b-906f42fb767d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:26.676669 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2966039-7092-49d7-bb9b-906f42fb767d admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:26.676948 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2966039-7092-49d7-bb9b-906f42fb767d admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:26.677222 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2966039-7092-49d7-bb9b-906f42fb767d admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:26.677851 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2966039-7092-49d7-bb9b-906f42fb767d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:26.680076 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2966039-7092-49d7-bb9b-906f42fb767d admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:26.710264 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 940/1860] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 9 12:13:26 2022] POST /identity/v3/projects => generated 375 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:13:26.714745 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494ca446-48ea-4c98-a1a3-2672f23bc3cc None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:26.725573 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494ca446-48ea-4c98-a1a3-2672f23bc3cc None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:26.783820 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004aca2c-7a49-4cf3-bfed-437989692215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:26.784376 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004aca2c-7a49-4cf3-bfed-437989692215 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:26.784615 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004aca2c-7a49-4cf3-bfed-437989692215 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:26.784881 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004aca2c-7a49-4cf3-bfed-437989692215 admin admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:26.785545 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004aca2c-7a49-4cf3-bfed-437989692215 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:26.787823 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004aca2c-7a49-4cf3-bfed-437989692215 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:26.809849 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 921/1861] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:13:26 2022] POST /identity/v3/users => generated 288 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:13:26.814036 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2966039-7092-49d7-bb9b-906f42fb767d admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:26.828729 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2966039-7092-49d7-bb9b-906f42fb767d admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:26.835361 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8c9f9d-9933-4fdf-8f6b-0ab52feb6b2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:26.835984 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8c9f9d-9933-4fdf-8f6b-0ab52feb6b2a admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:26.836267 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8c9f9d-9933-4fdf-8f6b-0ab52feb6b2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:26.836537 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8c9f9d-9933-4fdf-8f6b-0ab52feb6b2a admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:26.837264 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8c9f9d-9933-4fdf-8f6b-0ab52feb6b2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:26.839019 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8c9f9d-9933-4fdf-8f6b-0ab52feb6b2a admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:26.851797 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 941/1862] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:13:26 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:13:26.857291 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004aca2c-7a49-4cf3-bfed-437989692215 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:26.867645 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004aca2c-7a49-4cf3-bfed-437989692215 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:26.874744 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fa9d84-7e57-4e44-958b-ed9edbcd3c35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:26.875384 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fa9d84-7e57-4e44-958b-ed9edbcd3c35 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:26.875658 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fa9d84-7e57-4e44-958b-ed9edbcd3c35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:26.875926 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fa9d84-7e57-4e44-958b-ed9edbcd3c35 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:26.876565 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fa9d84-7e57-4e44-958b-ed9edbcd3c35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:26.878218 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fa9d84-7e57-4e44-958b-ed9edbcd3c35 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:26.889217 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 922/1863] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:13:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:13:26.894927 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8c9f9d-9933-4fdf-8f6b-0ab52feb6b2a admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:26.906974 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8c9f9d-9933-4fdf-8f6b-0ab52feb6b2a admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:26.911705 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0735f8-3caa-4021-9273-192614873071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:26.913741 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0735f8-3caa-4021-9273-192614873071 admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:26.913956 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0735f8-3caa-4021-9273-192614873071 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:26.915052 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0735f8-3caa-4021-9273-192614873071 admin admin] PATH_INFO: `/v3/projects/5c608cc0edad4f008d22a1cb1fcf984f/users/314f0613dd2a497f93f05c96afa41bb0/roles/eecd2f160dd844f5906f3d345780c501` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:26.933446 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0735f8-3caa-4021-9273-192614873071 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c608cc0edad4f008d22a1cb1fcf984f, user_id=314f0613dd2a497f93f05c96afa41bb0, role_id=eecd2f160dd844f5906f3d345780c501)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:26.938452 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0735f8-3caa-4021-9273-192614873071 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:26.958977 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 942/1864] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:13:26 2022] PUT /identity/v3/projects/5c608cc0edad4f008d22a1cb1fcf984f/users/314f0613dd2a497f93f05c96afa41bb0/roles/eecd2f160dd844f5906f3d345780c501 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:13:26.968734 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff947c29-7c16-48b0-ac93-b4c7f3c7c7b1 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:26.969029 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff947c29-7c16-48b0-ac93-b4c7f3c7c7b1 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:26.969295 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff947c29-7c16-48b0-ac93-b4c7f3c7c7b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:27.012046 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-ff947c29-7c16-48b0-ac93-b4c7f3c7c7b1 None None] MFA Rules not processed for user `314f0613dd2a497f93f05c96afa41bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:13:27.013386 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-ff947c29-7c16-48b0-ac93-b4c7f3c7c7b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:27.141769 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 923/1865] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:13:26 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:13:27.150170 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0735f8-3caa-4021-9273-192614873071 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:27.235185 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0735f8-3caa-4021-9273-192614873071 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:27.243947 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20247f7a-1576-4fd4-87ca-3f5e079c8257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:27.244955 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20247f7a-1576-4fd4-87ca-3f5e079c8257 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:27.245425 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20247f7a-1576-4fd4-87ca-3f5e079c8257 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:27.245885 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20247f7a-1576-4fd4-87ca-3f5e079c8257 admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:27.247031 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20247f7a-1576-4fd4-87ca-3f5e079c8257 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:27.249855 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20247f7a-1576-4fd4-87ca-3f5e079c8257 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:27.307277 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 943/1866] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 9 12:13:27 2022] POST /identity/v3/projects => generated 379 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:13:27.312370 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff947c29-7c16-48b0-ac93-b4c7f3c7c7b1 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:27.328147 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff947c29-7c16-48b0-ac93-b4c7f3c7c7b1 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:27.404980 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fe444f-b575-4965-bfe7-8a8537e07b35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:27.405721 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fe444f-b575-4965-bfe7-8a8537e07b35 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:27.406326 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fe444f-b575-4965-bfe7-8a8537e07b35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:27.407062 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fe444f-b575-4965-bfe7-8a8537e07b35 admin admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:27.407928 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fe444f-b575-4965-bfe7-8a8537e07b35 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:27.409875 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fe444f-b575-4965-bfe7-8a8537e07b35 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:27.423185 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:27.442609 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 924/1867] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:13:27 2022] POST /identity/v3/users => generated 296 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:13:27.443566 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20247f7a-1576-4fd4-87ca-3f5e079c8257 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:27.455536 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20247f7a-1576-4fd4-87ca-3f5e079c8257 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:27.462451 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1155280-044d-4f8f-8ca4-830b3299ed8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:27.463195 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1155280-044d-4f8f-8ca4-830b3299ed8d admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:27.463499 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1155280-044d-4f8f-8ca4-830b3299ed8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:27.463874 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1155280-044d-4f8f-8ca4-830b3299ed8d admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:27.464549 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1155280-044d-4f8f-8ca4-830b3299ed8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:27.466174 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1155280-044d-4f8f-8ca4-830b3299ed8d admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:27.479340 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 944/1868] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:13:27 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:13:27.483522 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fe444f-b575-4965-bfe7-8a8537e07b35 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:27.494847 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fe444f-b575-4965-bfe7-8a8537e07b35 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:27.500948 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a76cc9-9580-4c24-8a2a-2487b66f3cf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:27.501619 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a76cc9-9580-4c24-8a2a-2487b66f3cf0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:27.501896 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a76cc9-9580-4c24-8a2a-2487b66f3cf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:27.502165 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a76cc9-9580-4c24-8a2a-2487b66f3cf0 admin admin] PATH_INFO: `/v3/projects/b1043b8d4f6f41e895cfa78ceae9e067/users/688ec137f2df4e6ea9a8ddd541410ba5/roles/3f2ca33902a04b6dbc2ef7d06d7289d3` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:27.522081 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a76cc9-9580-4c24-8a2a-2487b66f3cf0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1043b8d4f6f41e895cfa78ceae9e067, user_id=688ec137f2df4e6ea9a8ddd541410ba5, role_id=3f2ca33902a04b6dbc2ef7d06d7289d3)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:27.525222 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a76cc9-9580-4c24-8a2a-2487b66f3cf0 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:27.540061 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 925/1869] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:13:27 2022] PUT /identity/v3/projects/b1043b8d4f6f41e895cfa78ceae9e067/users/688ec137f2df4e6ea9a8ddd541410ba5/roles/3f2ca33902a04b6dbc2ef7d06d7289d3 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:13:27.544837 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1155280-044d-4f8f-8ca4-830b3299ed8d admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:27.550425 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:27.557799 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1155280-044d-4f8f-8ca4-830b3299ed8d admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:27.614559 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42f05ab-723d-48c5-bf7e-9db828848424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:27.615196 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42f05ab-723d-48c5-bf7e-9db828848424 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:27.615478 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42f05ab-723d-48c5-bf7e-9db828848424 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:27.615753 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42f05ab-723d-48c5-bf7e-9db828848424 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:27.616394 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42f05ab-723d-48c5-bf7e-9db828848424 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:27.618388 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42f05ab-723d-48c5-bf7e-9db828848424 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:27.629398 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 945/1870] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:13:27 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:13:27.633547 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a76cc9-9580-4c24-8a2a-2487b66f3cf0 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:27.644806 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a76cc9-9580-4c24-8a2a-2487b66f3cf0 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:27.650360 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40457569-ed05-4912-ac49-dd6542f14752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:27.651184 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40457569-ed05-4912-ac49-dd6542f14752 admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:27.651465 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40457569-ed05-4912-ac49-dd6542f14752 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:27.651737 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40457569-ed05-4912-ac49-dd6542f14752 admin admin] PATH_INFO: `/v3/projects/b1043b8d4f6f41e895cfa78ceae9e067/users/688ec137f2df4e6ea9a8ddd541410ba5/roles/3f2ca33902a04b6dbc2ef7d06d7289d3` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:27.653735 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40457569-ed05-4912-ac49-dd6542f14752 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1043b8d4f6f41e895cfa78ceae9e067, user_id=688ec137f2df4e6ea9a8ddd541410ba5, role_id=3f2ca33902a04b6dbc2ef7d06d7289d3)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:27.656458 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40457569-ed05-4912-ac49-dd6542f14752 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:27.668235 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 926/1871] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:13:27 2022] PUT /identity/v3/projects/b1043b8d4f6f41e895cfa78ceae9e067/users/688ec137f2df4e6ea9a8ddd541410ba5/roles/3f2ca33902a04b6dbc2ef7d06d7289d3 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:13:27.679382 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e9a005-ae26-45ed-8748-811ec630463d None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:27.679674 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e9a005-ae26-45ed-8748-811ec630463d None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:27.679950 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e9a005-ae26-45ed-8748-811ec630463d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:27.708021 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-61e9a005-ae26-45ed-8748-811ec630463d None None] MFA Rules not processed for user `688ec137f2df4e6ea9a8ddd541410ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:13:27.709283 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-61e9a005-ae26-45ed-8748-811ec630463d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:27.792725 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 946/1872] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:13:27 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:13:27.803885 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40457569-ed05-4912-ac49-dd6542f14752 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:27.805592 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-40457569-ed05-4912-ac49-dd6542f14752 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:27.865869 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40457569-ed05-4912-ac49-dd6542f14752 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:27.869731 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c077d4-45f1-40d0-8984-37969859dfb6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535c258100df40b58a5ee44b2c21d621', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:27.870360 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c077d4-45f1-40d0-8984-37969859dfb6 service cinder] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:27.870632 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c077d4-45f1-40d0-8984-37969859dfb6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:27.870920 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c077d4-45f1-40d0-8984-37969859dfb6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:27.879705 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c077d4-45f1-40d0-8984-37969859dfb6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:27.881180 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c077d4-45f1-40d0-8984-37969859dfb6 service cinder] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:27.890930 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 927/1873] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:13:27 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:13:27.932706 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e9a005-ae26-45ed-8748-811ec630463d None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:27.934121 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-61e9a005-ae26-45ed-8748-811ec630463d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:27.945999 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e9a005-ae26-45ed-8748-811ec630463d None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:27.949724 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79473fa-151e-4ed7-b699-545531f53726 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535c258100df40b58a5ee44b2c21d621', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:27.950322 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79473fa-151e-4ed7-b699-545531f53726 service cinder] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:27.950597 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79473fa-151e-4ed7-b699-545531f53726 service cinder] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:27.950864 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79473fa-151e-4ed7-b699-545531f53726 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:28.007198 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79473fa-151e-4ed7-b699-545531f53726 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:28.008631 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79473fa-151e-4ed7-b699-545531f53726 service cinder] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:28.018044 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 947/1874] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:13:27 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:13:32.496025 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c077d4-45f1-40d0-8984-37969859dfb6 service cinder] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:32.499213 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-98c077d4-45f1-40d0-8984-37969859dfb6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:32.612148 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c077d4-45f1-40d0-8984-37969859dfb6 service cinder] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:32.633175 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bcd8ae-a3c4-4029-8296-cefef14c87d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:32.633959 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bcd8ae-a3c4-4029-8296-cefef14c87d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:32.634500 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bcd8ae-a3c4-4029-8296-cefef14c87d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:32.635959 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bcd8ae-a3c4-4029-8296-cefef14c87d4 admin admin] PATH_INFO: `/v3/users/9993833b9e3648b6b4f256a3eb6e358d` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:32.636402 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bcd8ae-a3c4-4029-8296-cefef14c87d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9993833b9e3648b6b4f256a3eb6e358d)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:32.639893 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bcd8ae-a3c4-4029-8296-cefef14c87d4 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:32.749208 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-57bcd8ae-a3c4-4029-8296-cefef14c87d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9993833b9e3648b6b4f256a3eb6e358d'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:32.788812 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:32.793039 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-57bcd8ae-a3c4-4029-8296-cefef14c87d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9993833b9e3648b6b4f256a3eb6e358d'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:32.817196 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-57bcd8ae-a3c4-4029-8296-cefef14c87d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9993833b9e3648b6b4f256a3eb6e358d'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:32.832945 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-57bcd8ae-a3c4-4029-8296-cefef14c87d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9993833b9e3648b6b4f256a3eb6e358d'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:32.833757 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-57bcd8ae-a3c4-4029-8296-cefef14c87d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9993833b9e3648b6b4f256a3eb6e358d'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:32.841705 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 928/1875] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 9 12:13:32 2022] DELETE /identity/v3/users/9993833b9e3648b6b4f256a3eb6e358d => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:13:32.846568 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476ff2b2-8dd2-40fd-a319-45e36e7e96b3 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:32.847627 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476ff2b2-8dd2-40fd-a319-45e36e7e96b3 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:32.848191 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476ff2b2-8dd2-40fd-a319-45e36e7e96b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:32.882395 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-476ff2b2-8dd2-40fd-a319-45e36e7e96b3 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:13:32.886579 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-476ff2b2-8dd2-40fd-a319-45e36e7e96b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:32.970489 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:33.110399 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 948/1876] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:13:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:13:33.136261 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bcd8ae-a3c4-4029-8296-cefef14c87d4 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:33.139806 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-57bcd8ae-a3c4-4029-8296-cefef14c87d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:33.235873 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bcd8ae-a3c4-4029-8296-cefef14c87d4 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:33.240893 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f54e380-6e12-487a-bd06-44a12dc94c43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:33.241840 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f54e380-6e12-487a-bd06-44a12dc94c43 service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:33.242333 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f54e380-6e12-487a-bd06-44a12dc94c43 service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:33.242781 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f54e380-6e12-487a-bd06-44a12dc94c43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:33.256591 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f54e380-6e12-487a-bd06-44a12dc94c43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:33.259455 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f54e380-6e12-487a-bd06-44a12dc94c43 service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:33.273168 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 929/1877] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:13:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:13:33.572779 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476ff2b2-8dd2-40fd-a319-45e36e7e96b3 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:33.574388 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-476ff2b2-8dd2-40fd-a319-45e36e7e96b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:33.596601 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476ff2b2-8dd2-40fd-a319-45e36e7e96b3 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:33.601745 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25ed633-4ec3-4cba-9390-bcce948c25d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:33.603070 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25ed633-4ec3-4cba-9390-bcce948c25d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:33.603482 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25ed633-4ec3-4cba-9390-bcce948c25d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:33.603827 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25ed633-4ec3-4cba-9390-bcce948c25d3 admin admin] PATH_INFO: `/v3/projects/8aa271ab4bd94e6f8269f67601936ad9` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:33.605182 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25ed633-4ec3-4cba-9390-bcce948c25d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8aa271ab4bd94e6f8269f67601936ad9)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:33.608278 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25ed633-4ec3-4cba-9390-bcce948c25d3 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:33.665135 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-d25ed633-4ec3-4cba-9390-bcce948c25d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8aa271ab4bd94e6f8269f67601936ad9', 'request_id': 'req-d25ed633-4ec3-4cba-9390-bcce948c25d3'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:33.671459 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-d25ed633-4ec3-4cba-9390-bcce948c25d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8aa271ab4bd94e6f8269f67601936ad9', 'request_id': 'req-d25ed633-4ec3-4cba-9390-bcce948c25d3'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:33.678266 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-d25ed633-4ec3-4cba-9390-bcce948c25d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8aa271ab4bd94e6f8269f67601936ad9', 'request_id': 'req-d25ed633-4ec3-4cba-9390-bcce948c25d3'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:33.692033 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-d25ed633-4ec3-4cba-9390-bcce948c25d3 admin admin] The token cache is being invalidate because project 8aa271ab4bd94e6f8269f67601936ad9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81591) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:13:33.692348 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-d25ed633-4ec3-4cba-9390-bcce948c25d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:33.692935 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-d25ed633-4ec3-4cba-9390-bcce948c25d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:33.694667 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 949/1878] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 9 12:13:33 2022] DELETE /identity/v3/projects/8aa271ab4bd94e6f8269f67601936ad9 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:13:33.699165 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f54e380-6e12-487a-bd06-44a12dc94c43 service neutron] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:33.708927 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f54e380-6e12-487a-bd06-44a12dc94c43 service neutron] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:33.773460 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adadcd9-a783-44f5-9bc0-4a04d3ba5075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:33.774094 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adadcd9-a783-44f5-9bc0-4a04d3ba5075 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:33.774500 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adadcd9-a783-44f5-9bc0-4a04d3ba5075 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:33.774889 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adadcd9-a783-44f5-9bc0-4a04d3ba5075 admin admin] PATH_INFO: `/v3/users/89a9e2a6b5004f5fbac737b7c538fb69` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:33.776169 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adadcd9-a783-44f5-9bc0-4a04d3ba5075 admin admin] RBAC: Authorizing `identity:delete_user(user_id=89a9e2a6b5004f5fbac737b7c538fb69)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:33.778093 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adadcd9-a783-44f5-9bc0-4a04d3ba5075 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:33.824774 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-1adadcd9-a783-44f5-9bc0-4a04d3ba5075 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89a9e2a6b5004f5fbac737b7c538fb69'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:33.836594 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-1adadcd9-a783-44f5-9bc0-4a04d3ba5075 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89a9e2a6b5004f5fbac737b7c538fb69'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:33.844668 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-1adadcd9-a783-44f5-9bc0-4a04d3ba5075 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89a9e2a6b5004f5fbac737b7c538fb69'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:33.851920 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-1adadcd9-a783-44f5-9bc0-4a04d3ba5075 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89a9e2a6b5004f5fbac737b7c538fb69'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:33.852615 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-1adadcd9-a783-44f5-9bc0-4a04d3ba5075 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89a9e2a6b5004f5fbac737b7c538fb69'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:33.858485 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 930/1879] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 9 12:13:33 2022] DELETE /identity/v3/users/89a9e2a6b5004f5fbac737b7c538fb69 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:13:34.162985 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25ed633-4ec3-4cba-9390-bcce948c25d3 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:34.164534 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-d25ed633-4ec3-4cba-9390-bcce948c25d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:34.179334 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25ed633-4ec3-4cba-9390-bcce948c25d3 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:34.258581 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d66aa0-b876-4ee4-aee7-51f4fe0bbf86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:34.259464 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d66aa0-b876-4ee4-aee7-51f4fe0bbf86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:34.259740 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d66aa0-b876-4ee4-aee7-51f4fe0bbf86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:34.260013 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d66aa0-b876-4ee4-aee7-51f4fe0bbf86 admin admin] PATH_INFO: `/v3/projects/bb91af272330489790ddcae6cc82f2ab` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:34.261124 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d66aa0-b876-4ee4-aee7-51f4fe0bbf86 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb91af272330489790ddcae6cc82f2ab)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:34.263039 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d66aa0-b876-4ee4-aee7-51f4fe0bbf86 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:34.311269 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:34.312651 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-38d66aa0-b876-4ee4-aee7-51f4fe0bbf86 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb91af272330489790ddcae6cc82f2ab', 'request_id': 'req-38d66aa0-b876-4ee4-aee7-51f4fe0bbf86'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:34.320163 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-38d66aa0-b876-4ee4-aee7-51f4fe0bbf86 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb91af272330489790ddcae6cc82f2ab', 'request_id': 'req-38d66aa0-b876-4ee4-aee7-51f4fe0bbf86'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:34.327561 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-38d66aa0-b876-4ee4-aee7-51f4fe0bbf86 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb91af272330489790ddcae6cc82f2ab', 'request_id': 'req-38d66aa0-b876-4ee4-aee7-51f4fe0bbf86'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:34.342691 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-38d66aa0-b876-4ee4-aee7-51f4fe0bbf86 admin admin] The token cache is being invalidate because project bb91af272330489790ddcae6cc82f2ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81591) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:13:34.343010 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-38d66aa0-b876-4ee4-aee7-51f4fe0bbf86 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:34.343675 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-38d66aa0-b876-4ee4-aee7-51f4fe0bbf86 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:34.345308 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 950/1880] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 9 12:13:34 2022] DELETE /identity/v3/projects/bb91af272330489790ddcae6cc82f2ab => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:13:34.351490 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81fdca2-5944-4862-b882-c429c961f5a2 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:34.351776 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81fdca2-5944-4862-b882-c429c961f5a2 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:34.352056 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81fdca2-5944-4862-b882-c429c961f5a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:34.363464 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:34.379658 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-c81fdca2-5944-4862-b882-c429c961f5a2 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:13:34.382006 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-c81fdca2-5944-4862-b882-c429c961f5a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:34.448201 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 931/1881] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:13:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:13:34.455984 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acca238-c2f6-4c9e-b759-00a447c8d4e6 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:34.456339 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acca238-c2f6-4c9e-b759-00a447c8d4e6 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:34.456681 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acca238-c2f6-4c9e-b759-00a447c8d4e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:34.476993 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-6acca238-c2f6-4c9e-b759-00a447c8d4e6 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:13:34.479355 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-6acca238-c2f6-4c9e-b759-00a447c8d4e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:34.487297 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 951/1882] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:13:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:13:34.491589 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81fdca2-5944-4862-b882-c429c961f5a2 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:34.501186 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81fdca2-5944-4862-b882-c429c961f5a2 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:34.505820 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1f7e35-d965-4a77-b9d5-73998f42d816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:34.506428 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1f7e35-d965-4a77-b9d5-73998f42d816 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:34.506706 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1f7e35-d965-4a77-b9d5-73998f42d816 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:34.507013 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1f7e35-d965-4a77-b9d5-73998f42d816 admin admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:34.507658 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1f7e35-d965-4a77-b9d5-73998f42d816 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:13:34.508040 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1f7e35-d965-4a77-b9d5-73998f42d816 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:34.509646 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1f7e35-d965-4a77-b9d5-73998f42d816 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:34.524837 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 932/1883] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 9 12:13:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:13:34.530543 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acca238-c2f6-4c9e-b759-00a447c8d4e6 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:34.543856 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acca238-c2f6-4c9e-b759-00a447c8d4e6 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:34.548408 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bbaa4a-5b23-455c-b041-6e97067bfa69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:34.549000 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bbaa4a-5b23-455c-b041-6e97067bfa69 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:34.549277 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bbaa4a-5b23-455c-b041-6e97067bfa69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:34.549541 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bbaa4a-5b23-455c-b041-6e97067bfa69 admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:34.550158 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bbaa4a-5b23-455c-b041-6e97067bfa69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:34.552078 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bbaa4a-5b23-455c-b041-6e97067bfa69 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:34.579083 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 952/1884] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 9 12:13:34 2022] POST /identity/v3/projects => generated 397 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:13:34.583223 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1f7e35-d965-4a77-b9d5-73998f42d816 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:34.593753 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1f7e35-d965-4a77-b9d5-73998f42d816 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:34.651915 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a304828-7339-453f-bcdb-3e87f8b4a340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:34.652519 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a304828-7339-453f-bcdb-3e87f8b4a340 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:34.652819 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a304828-7339-453f-bcdb-3e87f8b4a340 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:34.653109 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a304828-7339-453f-bcdb-3e87f8b4a340 admin admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:34.653747 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a304828-7339-453f-bcdb-3e87f8b4a340 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:34.655683 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a304828-7339-453f-bcdb-3e87f8b4a340 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:34.677739 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 933/1885] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:13:34 2022] POST /identity/v3/users => generated 299 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:13:34.682294 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bbaa4a-5b23-455c-b041-6e97067bfa69 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:34.692902 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bbaa4a-5b23-455c-b041-6e97067bfa69 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:34.697530 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74812eb5-db5c-4aa9-a7dd-02ab2ba6c253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:34.698120 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74812eb5-db5c-4aa9-a7dd-02ab2ba6c253 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:34.698396 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74812eb5-db5c-4aa9-a7dd-02ab2ba6c253 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:34.698663 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74812eb5-db5c-4aa9-a7dd-02ab2ba6c253 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:34.699326 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74812eb5-db5c-4aa9-a7dd-02ab2ba6c253 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:34.700934 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74812eb5-db5c-4aa9-a7dd-02ab2ba6c253 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:34.710550 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 953/1886] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:13:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:13:34.714874 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a304828-7339-453f-bcdb-3e87f8b4a340 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:34.724797 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a304828-7339-453f-bcdb-3e87f8b4a340 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:34.730099 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d86b80-0b07-428a-95b6-ea0144c4a100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:34.730690 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d86b80-0b07-428a-95b6-ea0144c4a100 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:34.730994 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d86b80-0b07-428a-95b6-ea0144c4a100 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:34.731265 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d86b80-0b07-428a-95b6-ea0144c4a100 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:34.731896 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d86b80-0b07-428a-95b6-ea0144c4a100 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:34.733510 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d86b80-0b07-428a-95b6-ea0144c4a100 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:34.744181 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 934/1887] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:13:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:13:34.748165 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74812eb5-db5c-4aa9-a7dd-02ab2ba6c253 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:34.758019 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74812eb5-db5c-4aa9-a7dd-02ab2ba6c253 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:34.763296 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681e494d-4145-482a-990d-9bedea7fc308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:34.763878 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681e494d-4145-482a-990d-9bedea7fc308 admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:34.764155 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681e494d-4145-482a-990d-9bedea7fc308 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:34.764425 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681e494d-4145-482a-990d-9bedea7fc308 admin admin] PATH_INFO: `/v3/projects/f555ce77fce8415888ae4011f760532f/users/f73a124bc3e64e338a7c4542a69c6bb8/roles/eecd2f160dd844f5906f3d345780c501` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:34.780560 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681e494d-4145-482a-990d-9bedea7fc308 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f555ce77fce8415888ae4011f760532f, user_id=f73a124bc3e64e338a7c4542a69c6bb8, role_id=eecd2f160dd844f5906f3d345780c501)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:34.783477 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681e494d-4145-482a-990d-9bedea7fc308 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:34.798459 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 954/1888] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:13:34 2022] PUT /identity/v3/projects/f555ce77fce8415888ae4011f760532f/users/f73a124bc3e64e338a7c4542a69c6bb8/roles/eecd2f160dd844f5906f3d345780c501 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:13:34.808632 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f5d9f8-c1c0-4aa3-b7f3-86d78721a704 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:34.808917 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f5d9f8-c1c0-4aa3-b7f3-86d78721a704 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:34.809189 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f5d9f8-c1c0-4aa3-b7f3-86d78721a704 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:34.839487 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-37f5d9f8-c1c0-4aa3-b7f3-86d78721a704 None None] MFA Rules not processed for user `f73a124bc3e64e338a7c4542a69c6bb8`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:13:34.840722 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-37f5d9f8-c1c0-4aa3-b7f3-86d78721a704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:34.903869 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:34.910439 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 935/1889] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:13:34 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:13:34.921595 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681e494d-4145-482a-990d-9bedea7fc308 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:34.923338 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-681e494d-4145-482a-990d-9bedea7fc308 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:34.986481 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681e494d-4145-482a-990d-9bedea7fc308 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:34.990176 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b2f5c7-b2b2-4b81-9a47-2e67f9a71975 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535c258100df40b58a5ee44b2c21d621', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:34.990787 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b2f5c7-b2b2-4b81-9a47-2e67f9a71975 service cinder] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:34.991099 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b2f5c7-b2b2-4b81-9a47-2e67f9a71975 service cinder] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:34.991378 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b2f5c7-b2b2-4b81-9a47-2e67f9a71975 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:35.000092 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b2f5c7-b2b2-4b81-9a47-2e67f9a71975 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:35.001523 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b2f5c7-b2b2-4b81-9a47-2e67f9a71975 service cinder] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:35.020432 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 955/1890] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:13:34 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:13:36.734636 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f5d9f8-c1c0-4aa3-b7f3-86d78721a704 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:36.736572 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-37f5d9f8-c1c0-4aa3-b7f3-86d78721a704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:36.850253 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f5d9f8-c1c0-4aa3-b7f3-86d78721a704 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:36.857024 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9ab348-71ae-42ec-81c5-804e43f33151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:36.857813 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9ab348-71ae-42ec-81c5-804e43f33151 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:36.858157 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9ab348-71ae-42ec-81c5-804e43f33151 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:36.858498 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9ab348-71ae-42ec-81c5-804e43f33151 admin admin] PATH_INFO: `/v3/users/c5b39da495d64c5dbea3aea312f31586` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:36.860060 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9ab348-71ae-42ec-81c5-804e43f33151 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5b39da495d64c5dbea3aea312f31586)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:36.862866 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9ab348-71ae-42ec-81c5-804e43f33151 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:36.926605 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-cb9ab348-71ae-42ec-81c5-804e43f33151 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5b39da495d64c5dbea3aea312f31586'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:36.941278 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-cb9ab348-71ae-42ec-81c5-804e43f33151 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5b39da495d64c5dbea3aea312f31586'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:36.949671 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-cb9ab348-71ae-42ec-81c5-804e43f33151 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5b39da495d64c5dbea3aea312f31586'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:36.958805 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-cb9ab348-71ae-42ec-81c5-804e43f33151 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5b39da495d64c5dbea3aea312f31586'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:36.959589 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-cb9ab348-71ae-42ec-81c5-804e43f33151 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5b39da495d64c5dbea3aea312f31586'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:36.967500 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 936/1891] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 9 12:13:36 2022] DELETE /identity/v3/users/c5b39da495d64c5dbea3aea312f31586 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:13:36.971822 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94ce34a-126b-4ad5-97f2-4c125ef7aa4f None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:36.972264 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94ce34a-126b-4ad5-97f2-4c125ef7aa4f None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:36.972371 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94ce34a-126b-4ad5-97f2-4c125ef7aa4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:37.005116 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-a94ce34a-126b-4ad5-97f2-4c125ef7aa4f None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:13:37.006340 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-a94ce34a-126b-4ad5-97f2-4c125ef7aa4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:37.070672 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 956/1892] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:13:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:13:37.080729 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9ab348-71ae-42ec-81c5-804e43f33151 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:37.082298 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-cb9ab348-71ae-42ec-81c5-804e43f33151 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:37.144902 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9ab348-71ae-42ec-81c5-804e43f33151 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:37.152820 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befc56a2-caee-405d-9e90-60e081bbe618 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:37.153444 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befc56a2-caee-405d-9e90-60e081bbe618 service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:37.153728 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befc56a2-caee-405d-9e90-60e081bbe618 service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:37.154001 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befc56a2-caee-405d-9e90-60e081bbe618 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:37.164490 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befc56a2-caee-405d-9e90-60e081bbe618 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:37.166307 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befc56a2-caee-405d-9e90-60e081bbe618 service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:37.170169 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94ce34a-126b-4ad5-97f2-4c125ef7aa4f None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:37.171770 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-a94ce34a-126b-4ad5-97f2-4c125ef7aa4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:37.176449 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 937/1893] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:13:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:13:37.183801 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94ce34a-126b-4ad5-97f2-4c125ef7aa4f None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:37.188806 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56094713-a2ac-4b91-8259-03293dedcd6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:37.189471 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56094713-a2ac-4b91-8259-03293dedcd6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:37.189812 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56094713-a2ac-4b91-8259-03293dedcd6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:37.191181 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56094713-a2ac-4b91-8259-03293dedcd6d admin admin] PATH_INFO: `/v3/users/6517ef70f96b4e089579b3f4cc4a1e91` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:37.192666 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56094713-a2ac-4b91-8259-03293dedcd6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=6517ef70f96b4e089579b3f4cc4a1e91)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:37.194672 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56094713-a2ac-4b91-8259-03293dedcd6d admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:37.207985 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:37.253895 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-56094713-a2ac-4b91-8259-03293dedcd6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6517ef70f96b4e089579b3f4cc4a1e91'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:37.261914 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-56094713-a2ac-4b91-8259-03293dedcd6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6517ef70f96b4e089579b3f4cc4a1e91'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:37.275966 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-56094713-a2ac-4b91-8259-03293dedcd6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6517ef70f96b4e089579b3f4cc4a1e91'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:37.285107 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-56094713-a2ac-4b91-8259-03293dedcd6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6517ef70f96b4e089579b3f4cc4a1e91'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:37.285912 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-56094713-a2ac-4b91-8259-03293dedcd6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6517ef70f96b4e089579b3f4cc4a1e91'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:37.295114 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 957/1894] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 9 12:13:37 2022] DELETE /identity/v3/users/6517ef70f96b4e089579b3f4cc4a1e91 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:13:37.299769 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453fe0da-45b6-4a4b-b501-f953f6b840e8 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:37.300416 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453fe0da-45b6-4a4b-b501-f953f6b840e8 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:37.301001 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453fe0da-45b6-4a4b-b501-f953f6b840e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:37.322464 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-453fe0da-45b6-4a4b-b501-f953f6b840e8 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:13:37.324031 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-453fe0da-45b6-4a4b-b501-f953f6b840e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:37.349388 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:37.408099 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 938/1895] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:13:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:13:37.418724 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56094713-a2ac-4b91-8259-03293dedcd6d admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:37.420296 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-56094713-a2ac-4b91-8259-03293dedcd6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:37.521833 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56094713-a2ac-4b91-8259-03293dedcd6d admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:37.527816 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b8f196-9d81-4711-a700-d12cb35fa64c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:37.528628 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b8f196-9d81-4711-a700-d12cb35fa64c service neutron] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:37.528902 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b8f196-9d81-4711-a700-d12cb35fa64c service neutron] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:37.529175 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b8f196-9d81-4711-a700-d12cb35fa64c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:37.538561 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453fe0da-45b6-4a4b-b501-f953f6b840e8 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:37.541356 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-453fe0da-45b6-4a4b-b501-f953f6b840e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:37.543207 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b8f196-9d81-4711-a700-d12cb35fa64c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:37.545685 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b8f196-9d81-4711-a700-d12cb35fa64c service neutron] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:37.560852 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 958/1896] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:13:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:13:37.561432 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453fe0da-45b6-4a4b-b501-f953f6b840e8 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:37.567337 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37febb0-565d-49a7-a129-17b68550ddbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:37.567991 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37febb0-565d-49a7-a129-17b68550ddbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:37.568267 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37febb0-565d-49a7-a129-17b68550ddbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:37.568543 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37febb0-565d-49a7-a129-17b68550ddbd admin admin] PATH_INFO: `/v3/projects/254937a622844299b5f0238075e81b1c` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:37.569689 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37febb0-565d-49a7-a129-17b68550ddbd admin admin] RBAC: Authorizing `identity:delete_project(project_id=254937a622844299b5f0238075e81b1c)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:37.572592 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37febb0-565d-49a7-a129-17b68550ddbd admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:37.632999 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-d37febb0-565d-49a7-a129-17b68550ddbd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '254937a622844299b5f0238075e81b1c', 'request_id': 'req-d37febb0-565d-49a7-a129-17b68550ddbd'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:37.648264 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-d37febb0-565d-49a7-a129-17b68550ddbd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '254937a622844299b5f0238075e81b1c', 'request_id': 'req-d37febb0-565d-49a7-a129-17b68550ddbd'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:37.657964 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-d37febb0-565d-49a7-a129-17b68550ddbd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '254937a622844299b5f0238075e81b1c', 'request_id': 'req-d37febb0-565d-49a7-a129-17b68550ddbd'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:37.671064 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-d37febb0-565d-49a7-a129-17b68550ddbd admin admin] The token cache is being invalidate because project 254937a622844299b5f0238075e81b1c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:13:37.671378 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-d37febb0-565d-49a7-a129-17b68550ddbd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:37.672002 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-d37febb0-565d-49a7-a129-17b68550ddbd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:37.676586 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 939/1897] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 9 12:13:37 2022] DELETE /identity/v3/projects/254937a622844299b5f0238075e81b1c => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:13:37.678138 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b8f196-9d81-4711-a700-d12cb35fa64c service neutron] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:37.691612 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b8f196-9d81-4711-a700-d12cb35fa64c service neutron] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:37.773880 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28de17b4-4281-496e-bcf2-e0fbef05b305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:37.775220 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28de17b4-4281-496e-bcf2-e0fbef05b305 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:37.775802 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28de17b4-4281-496e-bcf2-e0fbef05b305 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:37.776162 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28de17b4-4281-496e-bcf2-e0fbef05b305 admin admin] PATH_INFO: `/v3/users/314f0613dd2a497f93f05c96afa41bb0` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:37.777784 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28de17b4-4281-496e-bcf2-e0fbef05b305 admin admin] RBAC: Authorizing `identity:delete_user(user_id=314f0613dd2a497f93f05c96afa41bb0)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:37.781763 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28de17b4-4281-496e-bcf2-e0fbef05b305 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:37.844274 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:37.873537 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-28de17b4-4281-496e-bcf2-e0fbef05b305 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '314f0613dd2a497f93f05c96afa41bb0'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:37.882681 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-28de17b4-4281-496e-bcf2-e0fbef05b305 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '314f0613dd2a497f93f05c96afa41bb0'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:37.897298 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-28de17b4-4281-496e-bcf2-e0fbef05b305 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '314f0613dd2a497f93f05c96afa41bb0'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:37.908306 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-28de17b4-4281-496e-bcf2-e0fbef05b305 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '314f0613dd2a497f93f05c96afa41bb0'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:37.910589 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-28de17b4-4281-496e-bcf2-e0fbef05b305 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '314f0613dd2a497f93f05c96afa41bb0'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:37.922079 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 959/1898] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 9 12:13:37 2022] DELETE /identity/v3/users/314f0613dd2a497f93f05c96afa41bb0 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:13:38.228636 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37febb0-565d-49a7-a129-17b68550ddbd admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:38.231541 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-d37febb0-565d-49a7-a129-17b68550ddbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:38.265354 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37febb0-565d-49a7-a129-17b68550ddbd admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:38.354476 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28de17b4-4281-496e-bcf2-e0fbef05b305 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:38.355014 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4a51d8-7a05-4783-8f90-8a5cfd4e1c32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:38.355674 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4a51d8-7a05-4783-8f90-8a5cfd4e1c32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:38.355953 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4a51d8-7a05-4783-8f90-8a5cfd4e1c32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:38.356121 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-28de17b4-4281-496e-bcf2-e0fbef05b305 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:38.356318 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4a51d8-7a05-4783-8f90-8a5cfd4e1c32 admin admin] PATH_INFO: `/v3/projects/8d9e072a5a2c432e946180291b81e1b0` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:38.357386 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4a51d8-7a05-4783-8f90-8a5cfd4e1c32 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d9e072a5a2c432e946180291b81e1b0)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:38.359531 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4a51d8-7a05-4783-8f90-8a5cfd4e1c32 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:38.381758 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28de17b4-4281-496e-bcf2-e0fbef05b305 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:38.388495 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c182fbbd-c164-42ce-a984-13c6c9df56d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:38.389151 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c182fbbd-c164-42ce-a984-13c6c9df56d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:38.389437 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c182fbbd-c164-42ce-a984-13c6c9df56d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:38.389755 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c182fbbd-c164-42ce-a984-13c6c9df56d3 admin admin] PATH_INFO: `/v3/projects/5c608cc0edad4f008d22a1cb1fcf984f` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:38.390915 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c182fbbd-c164-42ce-a984-13c6c9df56d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c608cc0edad4f008d22a1cb1fcf984f)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:38.393719 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c182fbbd-c164-42ce-a984-13c6c9df56d3 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:38.409573 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:38.413087 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-aa4a51d8-7a05-4783-8f90-8a5cfd4e1c32 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d9e072a5a2c432e946180291b81e1b0', 'request_id': 'req-aa4a51d8-7a05-4783-8f90-8a5cfd4e1c32'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:38.421269 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-aa4a51d8-7a05-4783-8f90-8a5cfd4e1c32 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d9e072a5a2c432e946180291b81e1b0', 'request_id': 'req-aa4a51d8-7a05-4783-8f90-8a5cfd4e1c32'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:38.428554 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-aa4a51d8-7a05-4783-8f90-8a5cfd4e1c32 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d9e072a5a2c432e946180291b81e1b0', 'request_id': 'req-aa4a51d8-7a05-4783-8f90-8a5cfd4e1c32'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:38.440721 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-aa4a51d8-7a05-4783-8f90-8a5cfd4e1c32 admin admin] The token cache is being invalidate because project 8d9e072a5a2c432e946180291b81e1b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:13:38.441011 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-aa4a51d8-7a05-4783-8f90-8a5cfd4e1c32 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:38.441651 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-aa4a51d8-7a05-4783-8f90-8a5cfd4e1c32 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:38.443144 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-c182fbbd-c164-42ce-a984-13c6c9df56d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c608cc0edad4f008d22a1cb1fcf984f', 'request_id': 'req-c182fbbd-c164-42ce-a984-13c6c9df56d3'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:38.443296 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 940/1899] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 9 12:13:38 2022] DELETE /identity/v3/projects/8d9e072a5a2c432e946180291b81e1b0 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:13:38.447978 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4a51d8-7a05-4783-8f90-8a5cfd4e1c32 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:38.449539 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-aa4a51d8-7a05-4783-8f90-8a5cfd4e1c32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:38.449723 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-c182fbbd-c164-42ce-a984-13c6c9df56d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c608cc0edad4f008d22a1cb1fcf984f', 'request_id': 'req-c182fbbd-c164-42ce-a984-13c6c9df56d3'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:38.456517 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-c182fbbd-c164-42ce-a984-13c6c9df56d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c608cc0edad4f008d22a1cb1fcf984f', 'request_id': 'req-c182fbbd-c164-42ce-a984-13c6c9df56d3'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:38.470531 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-c182fbbd-c164-42ce-a984-13c6c9df56d3 admin admin] The token cache is being invalidate because project 5c608cc0edad4f008d22a1cb1fcf984f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81591) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:13:38.470928 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-c182fbbd-c164-42ce-a984-13c6c9df56d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:38.471575 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-c182fbbd-c164-42ce-a984-13c6c9df56d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:38.473072 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 960/1900] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 9 12:13:38 2022] DELETE /identity/v3/projects/5c608cc0edad4f008d22a1cb1fcf984f => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:13:38.476969 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c182fbbd-c164-42ce-a984-13c6c9df56d3 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:38.478957 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-c182fbbd-c164-42ce-a984-13c6c9df56d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:38.522533 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4a51d8-7a05-4783-8f90-8a5cfd4e1c32 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:38.560681 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c182fbbd-c164-42ce-a984-13c6c9df56d3 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:38.567638 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3d3899-dbef-48e2-8b46-eff758113041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:38.568627 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3d3899-dbef-48e2-8b46-eff758113041 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:38.569083 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3d3899-dbef-48e2-8b46-eff758113041 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:38.569492 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3d3899-dbef-48e2-8b46-eff758113041 admin admin] PATH_INFO: `/v3/users/688ec137f2df4e6ea9a8ddd541410ba5` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:38.571130 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3d3899-dbef-48e2-8b46-eff758113041 admin admin] RBAC: Authorizing `identity:delete_user(user_id=688ec137f2df4e6ea9a8ddd541410ba5)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:38.573879 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3d3899-dbef-48e2-8b46-eff758113041 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:38.596875 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0e7925-026f-4053-bee6-746204c3c7c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:38.597511 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0e7925-026f-4053-bee6-746204c3c7c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:38.597791 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0e7925-026f-4053-bee6-746204c3c7c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:38.598058 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0e7925-026f-4053-bee6-746204c3c7c3 admin admin] PATH_INFO: `/v3/users/1b62daa8af5144499533f893cda3f57e` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:38.599286 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0e7925-026f-4053-bee6-746204c3c7c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b62daa8af5144499533f893cda3f57e)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:38.601216 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0e7925-026f-4053-bee6-746204c3c7c3 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:38.615491 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:38.630372 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-5c3d3899-dbef-48e2-8b46-eff758113041 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '688ec137f2df4e6ea9a8ddd541410ba5'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:38.638735 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-5c3d3899-dbef-48e2-8b46-eff758113041 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '688ec137f2df4e6ea9a8ddd541410ba5'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:38.652449 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-6f0e7925-026f-4053-bee6-746204c3c7c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b62daa8af5144499533f893cda3f57e'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:38.654219 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-5c3d3899-dbef-48e2-8b46-eff758113041 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '688ec137f2df4e6ea9a8ddd541410ba5'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:38.662784 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-5c3d3899-dbef-48e2-8b46-eff758113041 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '688ec137f2df4e6ea9a8ddd541410ba5'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:38.663491 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-5c3d3899-dbef-48e2-8b46-eff758113041 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '688ec137f2df4e6ea9a8ddd541410ba5'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:38.667466 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-6f0e7925-026f-4053-bee6-746204c3c7c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b62daa8af5144499533f893cda3f57e'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:38.670270 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 961/1901] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 9 12:13:38 2022] DELETE /identity/v3/users/688ec137f2df4e6ea9a8ddd541410ba5 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:13:38.674701 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-6f0e7925-026f-4053-bee6-746204c3c7c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b62daa8af5144499533f893cda3f57e'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:38.682657 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-6f0e7925-026f-4053-bee6-746204c3c7c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b62daa8af5144499533f893cda3f57e'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:38.683340 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-6f0e7925-026f-4053-bee6-746204c3c7c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b62daa8af5144499533f893cda3f57e'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:38.685977 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:38.691916 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 941/1902] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 9 12:13:38 2022] DELETE /identity/v3/users/1b62daa8af5144499533f893cda3f57e => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:13:39.028085 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3d3899-dbef-48e2-8b46-eff758113041 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:39.030132 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-5c3d3899-dbef-48e2-8b46-eff758113041 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:39.103947 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3d3899-dbef-48e2-8b46-eff758113041 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:39.171253 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f164b2-a333-4125-907c-4a5f8050a7e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:39.171899 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f164b2-a333-4125-907c-4a5f8050a7e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:39.172178 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f164b2-a333-4125-907c-4a5f8050a7e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:39.172450 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f164b2-a333-4125-907c-4a5f8050a7e7 admin admin] PATH_INFO: `/v3/projects/b1043b8d4f6f41e895cfa78ceae9e067` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:39.173574 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f164b2-a333-4125-907c-4a5f8050a7e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1043b8d4f6f41e895cfa78ceae9e067)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:39.175607 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f164b2-a333-4125-907c-4a5f8050a7e7 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:39.224978 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0e7925-026f-4053-bee6-746204c3c7c3 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:39.226719 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-6f0e7925-026f-4053-bee6-746204c3c7c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:39.242644 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0e7925-026f-4053-bee6-746204c3c7c3 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:39.245363 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-15f164b2-a333-4125-907c-4a5f8050a7e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1043b8d4f6f41e895cfa78ceae9e067', 'request_id': 'req-15f164b2-a333-4125-907c-4a5f8050a7e7'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:39.254476 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-15f164b2-a333-4125-907c-4a5f8050a7e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1043b8d4f6f41e895cfa78ceae9e067', 'request_id': 'req-15f164b2-a333-4125-907c-4a5f8050a7e7'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:39.256629 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:39.263184 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-15f164b2-a333-4125-907c-4a5f8050a7e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1043b8d4f6f41e895cfa78ceae9e067', 'request_id': 'req-15f164b2-a333-4125-907c-4a5f8050a7e7'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:39.280884 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-15f164b2-a333-4125-907c-4a5f8050a7e7 admin admin] The token cache is being invalidate because project b1043b8d4f6f41e895cfa78ceae9e067 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81591) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:13:39.281177 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-15f164b2-a333-4125-907c-4a5f8050a7e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:39.281954 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-15f164b2-a333-4125-907c-4a5f8050a7e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:39.284168 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 962/1903] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 9 12:13:39 2022] DELETE /identity/v3/projects/b1043b8d4f6f41e895cfa78ceae9e067 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:13:39.290323 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846a8b1f-0934-4c68-aa71-0f7616d9f3cc None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:39.290806 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846a8b1f-0934-4c68-aa71-0f7616d9f3cc None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:39.291119 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846a8b1f-0934-4c68-aa71-0f7616d9f3cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:39.312067 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ac1c4d-90ea-47fc-a40c-2dbe9892da8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:39.312885 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ac1c4d-90ea-47fc-a40c-2dbe9892da8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:39.313282 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ac1c4d-90ea-47fc-a40c-2dbe9892da8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:39.313616 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ac1c4d-90ea-47fc-a40c-2dbe9892da8c admin admin] PATH_INFO: `/v3/projects/1a295491c5374ce9a47d7970bc83b400` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:39.314150 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-846a8b1f-0934-4c68-aa71-0f7616d9f3cc None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:13:39.315080 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ac1c4d-90ea-47fc-a40c-2dbe9892da8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a295491c5374ce9a47d7970bc83b400)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:39.315535 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-846a8b1f-0934-4c68-aa71-0f7616d9f3cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:39.317075 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ac1c4d-90ea-47fc-a40c-2dbe9892da8c admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:39.325788 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 963/1904] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:13:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:13:39.333588 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c1f214-6da8-4d61-a3b5-8c509a5ed014 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:39.333868 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c1f214-6da8-4d61-a3b5-8c509a5ed014 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:39.334137 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c1f214-6da8-4d61-a3b5-8c509a5ed014 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:39.363638 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-58c1f214-6da8-4d61-a3b5-8c509a5ed014 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:13:39.365416 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-58c1f214-6da8-4d61-a3b5-8c509a5ed014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:39.377182 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 964/1905] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:13:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:13:39.378978 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c1f214-6da8-4d61-a3b5-8c509a5ed014 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:39.388546 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-58ac1c4d-90ea-47fc-a40c-2dbe9892da8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a295491c5374ce9a47d7970bc83b400', 'request_id': 'req-58ac1c4d-90ea-47fc-a40c-2dbe9892da8c'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:39.392263 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c1f214-6da8-4d61-a3b5-8c509a5ed014 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:39.396206 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-58ac1c4d-90ea-47fc-a40c-2dbe9892da8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a295491c5374ce9a47d7970bc83b400', 'request_id': 'req-58ac1c4d-90ea-47fc-a40c-2dbe9892da8c'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:39.406579 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-58ac1c4d-90ea-47fc-a40c-2dbe9892da8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a295491c5374ce9a47d7970bc83b400', 'request_id': 'req-58ac1c4d-90ea-47fc-a40c-2dbe9892da8c'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:39.420360 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-58ac1c4d-90ea-47fc-a40c-2dbe9892da8c admin admin] The token cache is being invalidate because project 1a295491c5374ce9a47d7970bc83b400 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:13:39.420653 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-58ac1c4d-90ea-47fc-a40c-2dbe9892da8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:39.421477 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-58ac1c4d-90ea-47fc-a40c-2dbe9892da8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:39.423215 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 942/1906] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 9 12:13:39 2022] DELETE /identity/v3/projects/1a295491c5374ce9a47d7970bc83b400 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:13:39.427689 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b00a9c9-99d1-458a-886e-60953e1d6359 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:39.427969 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b00a9c9-99d1-458a-886e-60953e1d6359 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:39.428238 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b00a9c9-99d1-458a-886e-60953e1d6359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:39.449849 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-1b00a9c9-99d1-458a-886e-60953e1d6359 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:13:39.452008 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-1b00a9c9-99d1-458a-886e-60953e1d6359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:39.458509 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f5778d-ecec-4797-ae89-95612b440efe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:39.459507 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f5778d-ecec-4797-ae89-95612b440efe admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:39.459858 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f5778d-ecec-4797-ae89-95612b440efe admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:39.460202 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f5778d-ecec-4797-ae89-95612b440efe admin admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:39.460920 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f5778d-ecec-4797-ae89-95612b440efe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:13:39.461542 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f5778d-ecec-4797-ae89-95612b440efe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:39.464618 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f5778d-ecec-4797-ae89-95612b440efe admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:39.480118 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 965/1907] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 9 12:13:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:13:39.485916 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f5778d-ecec-4797-ae89-95612b440efe admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:39.491680 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-08f5778d-ecec-4797-ae89-95612b440efe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:39.506082 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f5778d-ecec-4797-ae89-95612b440efe admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:39.512144 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1b7661-15f2-4594-b7ea-ea6dcbd5d2df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:39.512759 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1b7661-15f2-4594-b7ea-ea6dcbd5d2df admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:39.513049 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1b7661-15f2-4594-b7ea-ea6dcbd5d2df admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:39.513962 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1b7661-15f2-4594-b7ea-ea6dcbd5d2df admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:39.514796 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1b7661-15f2-4594-b7ea-ea6dcbd5d2df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:39.517182 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1b7661-15f2-4594-b7ea-ea6dcbd5d2df admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:39.534941 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 943/1908] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:13:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:13:39.541119 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9be2ca-e807-45e2-ad91-f0ec6a863e2f None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:39.541347 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9be2ca-e807-45e2-ad91-f0ec6a863e2f None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:39.541612 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9be2ca-e807-45e2-ad91-f0ec6a863e2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:39.552197 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 966/1909] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 9 12:13:39 2022] POST /identity/v3/projects => generated 375 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:13:39.556070 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1b7661-15f2-4594-b7ea-ea6dcbd5d2df admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:39.563078 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-ad9be2ca-e807-45e2-ad91-f0ec6a863e2f None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:13:39.564301 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-ad9be2ca-e807-45e2-ad91-f0ec6a863e2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:39.598387 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:39.626778 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1b7661-15f2-4594-b7ea-ea6dcbd5d2df admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:39.635509 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901378c1-872d-408f-af87-2e203db9cb70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:39.636082 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 944/1910] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:13:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:13:39.636190 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901378c1-872d-408f-af87-2e203db9cb70 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:39.636389 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901378c1-872d-408f-af87-2e203db9cb70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:39.636665 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901378c1-872d-408f-af87-2e203db9cb70 admin admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:39.637386 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901378c1-872d-408f-af87-2e203db9cb70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:39.639362 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901378c1-872d-408f-af87-2e203db9cb70 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:39.640214 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9be2ca-e807-45e2-ad91-f0ec6a863e2f None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:39.650458 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9be2ca-e807-45e2-ad91-f0ec6a863e2f None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:39.655649 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba06680-e2c8-4c52-9c21-a7ff6c058b23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:39.657940 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba06680-e2c8-4c52-9c21-a7ff6c058b23 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:39.658215 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba06680-e2c8-4c52-9c21-a7ff6c058b23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:39.658492 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba06680-e2c8-4c52-9c21-a7ff6c058b23 admin admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:39.659158 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba06680-e2c8-4c52-9c21-a7ff6c058b23 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:13:39.659453 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba06680-e2c8-4c52-9c21-a7ff6c058b23 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:39.661300 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba06680-e2c8-4c52-9c21-a7ff6c058b23 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:39.665804 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 967/1911] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:13:39 2022] POST /identity/v3/users => generated 288 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:13:39.669993 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901378c1-872d-408f-af87-2e203db9cb70 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:39.675146 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 945/1912] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 9 12:13:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:13:39.679189 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba06680-e2c8-4c52-9c21-a7ff6c058b23 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:39.679806 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901378c1-872d-408f-af87-2e203db9cb70 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:39.684514 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ad3160-263b-4efc-bb32-f1eee6edb767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:39.685140 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ad3160-263b-4efc-bb32-f1eee6edb767 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:39.685684 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ad3160-263b-4efc-bb32-f1eee6edb767 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:39.685955 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ad3160-263b-4efc-bb32-f1eee6edb767 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:39.686679 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ad3160-263b-4efc-bb32-f1eee6edb767 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:39.688330 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ad3160-263b-4efc-bb32-f1eee6edb767 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:39.689687 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba06680-e2c8-4c52-9c21-a7ff6c058b23 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:39.695988 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1698cd0a-7384-4a0e-a5b4-4b19dec67250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:39.696660 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1698cd0a-7384-4a0e-a5b4-4b19dec67250 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:39.696870 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1698cd0a-7384-4a0e-a5b4-4b19dec67250 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:39.697145 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1698cd0a-7384-4a0e-a5b4-4b19dec67250 admin admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:39.697774 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1698cd0a-7384-4a0e-a5b4-4b19dec67250 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:39.698464 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 968/1913] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:13:39 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:13:39.700263 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1698cd0a-7384-4a0e-a5b4-4b19dec67250 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:39.703509 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ad3160-263b-4efc-bb32-f1eee6edb767 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:39.713866 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ad3160-263b-4efc-bb32-f1eee6edb767 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:39.718872 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fff7fd2-be2b-472d-b1ef-79f739e714a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:39.719626 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fff7fd2-be2b-472d-b1ef-79f739e714a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:39.719929 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fff7fd2-be2b-472d-b1ef-79f739e714a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:39.720264 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fff7fd2-be2b-472d-b1ef-79f739e714a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:39.720924 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fff7fd2-be2b-472d-b1ef-79f739e714a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:39.722622 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fff7fd2-be2b-472d-b1ef-79f739e714a3 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:39.729861 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 946/1914] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 9 12:13:39 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:13:39.734020 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1698cd0a-7384-4a0e-a5b4-4b19dec67250 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:39.737289 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 969/1915] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:13:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:13:39.738198 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fff7fd2-be2b-472d-b1ef-79f739e714a3 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:39.749013 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fff7fd2-be2b-472d-b1ef-79f739e714a3 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:39.761009 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:39.809784 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1698cd0a-7384-4a0e-a5b4-4b19dec67250 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:39.815463 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe9a833-fcde-43bc-b5d9-8d595b41ce67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:39.816081 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe9a833-fcde-43bc-b5d9-8d595b41ce67 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:39.816419 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe9a833-fcde-43bc-b5d9-8d595b41ce67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:39.816634 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe9a833-fcde-43bc-b5d9-8d595b41ce67 admin admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:39.817278 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe9a833-fcde-43bc-b5d9-8d595b41ce67 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:39.819626 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe9a833-fcde-43bc-b5d9-8d595b41ce67 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:39.830427 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deda990d-32c3-4963-b4b3-10e73ffdfba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:39.831597 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deda990d-32c3-4963-b4b3-10e73ffdfba7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:39.831938 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deda990d-32c3-4963-b4b3-10e73ffdfba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:39.832393 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deda990d-32c3-4963-b4b3-10e73ffdfba7 admin admin] PATH_INFO: `/v3/projects/3b1160d8bfc942caab5d6841cbc695a1/users/71720a8873914d198d72a0ddd2c17deb/roles/eecd2f160dd844f5906f3d345780c501` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:39.850980 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 947/1916] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:13:39 2022] POST /identity/v3/users => generated 287 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:13:39.851346 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deda990d-32c3-4963-b4b3-10e73ffdfba7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b1160d8bfc942caab5d6841cbc695a1, user_id=71720a8873914d198d72a0ddd2c17deb, role_id=eecd2f160dd844f5906f3d345780c501)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:39.854984 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deda990d-32c3-4963-b4b3-10e73ffdfba7 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:39.857064 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe9a833-fcde-43bc-b5d9-8d595b41ce67 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:39.873386 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 970/1917] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:13:39 2022] PUT /identity/v3/projects/3b1160d8bfc942caab5d6841cbc695a1/users/71720a8873914d198d72a0ddd2c17deb/roles/eecd2f160dd844f5906f3d345780c501 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:13:39.879117 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe9a833-fcde-43bc-b5d9-8d595b41ce67 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:39.885789 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c096d080-4cda-489b-a2f6-eacf5b693b5f None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:39.886337 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c096d080-4cda-489b-a2f6-eacf5b693b5f None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:39.886618 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c096d080-4cda-489b-a2f6-eacf5b693b5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:39.919184 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-c096d080-4cda-489b-a2f6-eacf5b693b5f None None] MFA Rules not processed for user `71720a8873914d198d72a0ddd2c17deb`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:13:39.920601 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-c096d080-4cda-489b-a2f6-eacf5b693b5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:39.955753 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb42f46-9730-45b2-950c-08767c3b1808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:39.956732 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb42f46-9730-45b2-950c-08767c3b1808 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:39.957025 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb42f46-9730-45b2-950c-08767c3b1808 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:39.957419 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb42f46-9730-45b2-950c-08767c3b1808 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:39.958431 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb42f46-9730-45b2-950c-08767c3b1808 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:39.960078 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb42f46-9730-45b2-950c-08767c3b1808 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:39.975336 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 948/1918] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:13:39 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 12:13:39.976373 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb42f46-9730-45b2-950c-08767c3b1808 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:39.994847 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb42f46-9730-45b2-950c-08767c3b1808 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:40.000187 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f02ab00-6009-4efa-af36-b8b4cb515370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:40.000852 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f02ab00-6009-4efa-af36-b8b4cb515370 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:40.001205 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f02ab00-6009-4efa-af36-b8b4cb515370 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:40.001385 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f02ab00-6009-4efa-af36-b8b4cb515370 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:40.002022 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f02ab00-6009-4efa-af36-b8b4cb515370 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:40.003659 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f02ab00-6009-4efa-af36-b8b4cb515370 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:40.017860 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 949/1919] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:13:39 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:13:40.019220 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f02ab00-6009-4efa-af36-b8b4cb515370 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:40.031514 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 971/1920] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:13:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:13:40.036375 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f02ab00-6009-4efa-af36-b8b4cb515370 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:40.046194 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c096d080-4cda-489b-a2f6-eacf5b693b5f None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:40.048020 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-c096d080-4cda-489b-a2f6-eacf5b693b5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:40.049558 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587bae93-f092-4946-bf56-b16958fda16c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:40.050252 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587bae93-f092-4946-bf56-b16958fda16c admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:40.051403 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587bae93-f092-4946-bf56-b16958fda16c admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:40.051680 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587bae93-f092-4946-bf56-b16958fda16c admin admin] PATH_INFO: `/v3/projects/7ec24b38fc844648a4e61575b8ee1c70/users/073d857764e94ef0926214489742abee/roles/eecd2f160dd844f5906f3d345780c501` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:40.070548 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:40.083400 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587bae93-f092-4946-bf56-b16958fda16c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ec24b38fc844648a4e61575b8ee1c70, user_id=073d857764e94ef0926214489742abee, role_id=eecd2f160dd844f5906f3d345780c501)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:40.091012 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587bae93-f092-4946-bf56-b16958fda16c admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:40.113400 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 950/1921] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:13:40 2022] PUT /identity/v3/projects/7ec24b38fc844648a4e61575b8ee1c70/users/073d857764e94ef0926214489742abee/roles/eecd2f160dd844f5906f3d345780c501 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:13:40.126346 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f78c64f-0b4d-4b18-b21e-5a44cd12dde1 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:40.126637 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f78c64f-0b4d-4b18-b21e-5a44cd12dde1 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:40.126929 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f78c64f-0b4d-4b18-b21e-5a44cd12dde1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:40.210645 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-5f78c64f-0b4d-4b18-b21e-5a44cd12dde1 None None] MFA Rules not processed for user `073d857764e94ef0926214489742abee`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:13:40.216304 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-5f78c64f-0b4d-4b18-b21e-5a44cd12dde1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:40.217629 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c096d080-4cda-489b-a2f6-eacf5b693b5f None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:40.249439 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:40.307219 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be52fe02-f48b-4797-842f-ba2f51b20e2e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37c4e14aa8e04d3e85a406b9363bca2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:40.308133 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be52fe02-f48b-4797-842f-ba2f51b20e2e service glance] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:40.308664 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be52fe02-f48b-4797-842f-ba2f51b20e2e service glance] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:40.309612 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be52fe02-f48b-4797-842f-ba2f51b20e2e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:40.359256 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 951/1922] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:13:40 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:13:40.363901 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f78c64f-0b4d-4b18-b21e-5a44cd12dde1 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:40.425202 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be52fe02-f48b-4797-842f-ba2f51b20e2e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:40.431176 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be52fe02-f48b-4797-842f-ba2f51b20e2e service glance] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:40.452832 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 972/1923] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:13:40 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:13:40.498282 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f78c64f-0b4d-4b18-b21e-5a44cd12dde1 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:40.515612 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e224cb-5605-4eee-abd3-34199e5e4c8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:40.516308 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e224cb-5605-4eee-abd3-34199e5e4c8e admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:40.516578 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e224cb-5605-4eee-abd3-34199e5e4c8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:40.516842 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e224cb-5605-4eee-abd3-34199e5e4c8e admin admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:40.517533 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e224cb-5605-4eee-abd3-34199e5e4c8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:40.519539 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e224cb-5605-4eee-abd3-34199e5e4c8e admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:40.619726 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 952/1924] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 9 12:13:40 2022] POST /identity/v3/projects => generated 373 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:13:40.620486 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be52fe02-f48b-4797-842f-ba2f51b20e2e service glance] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:40.647549 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be52fe02-f48b-4797-842f-ba2f51b20e2e service glance] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:40.715265 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b55073-f2b4-4427-864e-80c2514380cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:40.716006 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b55073-f2b4-4427-864e-80c2514380cf admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:40.716384 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b55073-f2b4-4427-864e-80c2514380cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:40.716764 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b55073-f2b4-4427-864e-80c2514380cf admin admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:40.717572 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b55073-f2b4-4427-864e-80c2514380cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:40.720379 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b55073-f2b4-4427-864e-80c2514380cf admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:40.750240 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 973/1925] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:13:40 2022] POST /identity/v3/users => generated 293 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:13:40.751563 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e224cb-5605-4eee-abd3-34199e5e4c8e admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:40.762527 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e224cb-5605-4eee-abd3-34199e5e4c8e admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:40.767500 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b47e03-6d0d-46de-96ea-5945251649f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:40.768116 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b47e03-6d0d-46de-96ea-5945251649f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:40.768388 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b47e03-6d0d-46de-96ea-5945251649f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:40.768656 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b47e03-6d0d-46de-96ea-5945251649f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:40.769304 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b47e03-6d0d-46de-96ea-5945251649f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:40.770935 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b47e03-6d0d-46de-96ea-5945251649f4 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:40.782488 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 953/1926] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:13:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:13:40.786638 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b55073-f2b4-4427-864e-80c2514380cf admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:40.801723 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b55073-f2b4-4427-864e-80c2514380cf admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:40.809507 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7634e79-a24f-4d0d-a645-6f74b89193c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:40.810578 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7634e79-a24f-4d0d-a645-6f74b89193c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:40.810994 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7634e79-a24f-4d0d-a645-6f74b89193c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:40.811383 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7634e79-a24f-4d0d-a645-6f74b89193c3 admin admin] PATH_INFO: `/v3/projects/5d29cd398a894966b9c482938fad4196/users/17570e77a56643178fef6ff53dd83946/roles/3f2ca33902a04b6dbc2ef7d06d7289d3` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:40.828128 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7634e79-a24f-4d0d-a645-6f74b89193c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d29cd398a894966b9c482938fad4196, user_id=17570e77a56643178fef6ff53dd83946, role_id=3f2ca33902a04b6dbc2ef7d06d7289d3)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:40.839717 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7634e79-a24f-4d0d-a645-6f74b89193c3 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:40.856105 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 974/1927] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:13:40 2022] PUT /identity/v3/projects/5d29cd398a894966b9c482938fad4196/users/17570e77a56643178fef6ff53dd83946/roles/3f2ca33902a04b6dbc2ef7d06d7289d3 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:13:40.860752 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b47e03-6d0d-46de-96ea-5945251649f4 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:40.872590 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b47e03-6d0d-46de-96ea-5945251649f4 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:40.925584 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:40.937379 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504000f8-6884-4215-aff8-28b63f269602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:40.937987 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504000f8-6884-4215-aff8-28b63f269602 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:40.938262 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504000f8-6884-4215-aff8-28b63f269602 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:40.938531 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504000f8-6884-4215-aff8-28b63f269602 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:40.939202 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504000f8-6884-4215-aff8-28b63f269602 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:40.940804 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504000f8-6884-4215-aff8-28b63f269602 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:40.951519 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 954/1928] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:13:40 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:13:40.955979 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7634e79-a24f-4d0d-a645-6f74b89193c3 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:40.966550 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7634e79-a24f-4d0d-a645-6f74b89193c3 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:40.971118 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba724e9d-e8fa-458b-ba21-f90afb5219ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:40.971770 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba724e9d-e8fa-458b-ba21-f90afb5219ff admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:40.972070 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba724e9d-e8fa-458b-ba21-f90afb5219ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:40.972340 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba724e9d-e8fa-458b-ba21-f90afb5219ff admin admin] PATH_INFO: `/v3/projects/5d29cd398a894966b9c482938fad4196/users/17570e77a56643178fef6ff53dd83946/roles/3f2ca33902a04b6dbc2ef7d06d7289d3` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:40.974264 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba724e9d-e8fa-458b-ba21-f90afb5219ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d29cd398a894966b9c482938fad4196, user_id=17570e77a56643178fef6ff53dd83946, role_id=3f2ca33902a04b6dbc2ef7d06d7289d3)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:40.976943 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba724e9d-e8fa-458b-ba21-f90afb5219ff admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:40.994651 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 975/1929] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:13:40 2022] PUT /identity/v3/projects/5d29cd398a894966b9c482938fad4196/users/17570e77a56643178fef6ff53dd83946/roles/3f2ca33902a04b6dbc2ef7d06d7289d3 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:13:41.004794 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46613c6-3b83-4d4a-bc0d-78b322fa53a3 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:41.005091 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46613c6-3b83-4d4a-bc0d-78b322fa53a3 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:41.005369 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46613c6-3b83-4d4a-bc0d-78b322fa53a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:41.035100 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-b46613c6-3b83-4d4a-bc0d-78b322fa53a3 None None] MFA Rules not processed for user `17570e77a56643178fef6ff53dd83946`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:13:41.037045 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-b46613c6-3b83-4d4a-bc0d-78b322fa53a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:41.112068 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 955/1930] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:13:41 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:13:41.124664 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba724e9d-e8fa-458b-ba21-f90afb5219ff admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:41.126293 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-ba724e9d-e8fa-458b-ba21-f90afb5219ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:41.161463 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:41.188286 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba724e9d-e8fa-458b-ba21-f90afb5219ff admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:41.191892 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebd55c1-8c7c-4edb-9269-ac2efc8b24d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535c258100df40b58a5ee44b2c21d621', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:41.192495 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebd55c1-8c7c-4edb-9269-ac2efc8b24d3 service cinder] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:41.192771 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebd55c1-8c7c-4edb-9269-ac2efc8b24d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:41.193038 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebd55c1-8c7c-4edb-9269-ac2efc8b24d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:41.202264 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebd55c1-8c7c-4edb-9269-ac2efc8b24d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:41.203778 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebd55c1-8c7c-4edb-9269-ac2efc8b24d3 service cinder] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:41.213615 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 976/1931] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:13:41 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:13:41.610692 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46613c6-3b83-4d4a-bc0d-78b322fa53a3 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:41.686760 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46613c6-3b83-4d4a-bc0d-78b322fa53a3 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:41.691659 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5021db-f4ae-4760-9fea-abf11bbc7947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:41.692294 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5021db-f4ae-4760-9fea-abf11bbc7947 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:41.692571 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5021db-f4ae-4760-9fea-abf11bbc7947 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:41.692839 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5021db-f4ae-4760-9fea-abf11bbc7947 admin admin] PATH_INFO: `/v3/users/073d857764e94ef0926214489742abee` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:41.693996 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5021db-f4ae-4760-9fea-abf11bbc7947 admin admin] RBAC: Authorizing `identity:delete_user(user_id=073d857764e94ef0926214489742abee)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:41.696702 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5021db-f4ae-4760-9fea-abf11bbc7947 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:41.747631 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-1a5021db-f4ae-4760-9fea-abf11bbc7947 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '073d857764e94ef0926214489742abee'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:41.760877 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-1a5021db-f4ae-4760-9fea-abf11bbc7947 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '073d857764e94ef0926214489742abee'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:41.768319 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-1a5021db-f4ae-4760-9fea-abf11bbc7947 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '073d857764e94ef0926214489742abee'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:41.770154 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:41.777581 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-1a5021db-f4ae-4760-9fea-abf11bbc7947 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '073d857764e94ef0926214489742abee'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:41.778501 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-1a5021db-f4ae-4760-9fea-abf11bbc7947 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '073d857764e94ef0926214489742abee'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:41.785253 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 956/1932] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 9 12:13:41 2022] DELETE /identity/v3/users/073d857764e94ef0926214489742abee => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:13:41.790700 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d316fcb-1ac2-4ea9-ad0f-54ae289355c2 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:41.791210 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d316fcb-1ac2-4ea9-ad0f-54ae289355c2 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:41.791301 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d316fcb-1ac2-4ea9-ad0f-54ae289355c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:41.819670 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-3d316fcb-1ac2-4ea9-ad0f-54ae289355c2 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:13:41.821533 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-3d316fcb-1ac2-4ea9-ad0f-54ae289355c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:41.885646 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 977/1933] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:13:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:13:41.896029 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5021db-f4ae-4760-9fea-abf11bbc7947 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:41.897315 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-1a5021db-f4ae-4760-9fea-abf11bbc7947 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:41.958722 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5021db-f4ae-4760-9fea-abf11bbc7947 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:41.962880 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba85df0-36e0-4905-9cc6-d2fc09804101 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:41.963524 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba85df0-36e0-4905-9cc6-d2fc09804101 service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:41.963806 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba85df0-36e0-4905-9cc6-d2fc09804101 service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:41.964080 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba85df0-36e0-4905-9cc6-d2fc09804101 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:41.973545 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba85df0-36e0-4905-9cc6-d2fc09804101 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:41.975068 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba85df0-36e0-4905-9cc6-d2fc09804101 service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:41.986282 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 957/1934] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:13:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:13:42.340664 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d316fcb-1ac2-4ea9-ad0f-54ae289355c2 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:42.342082 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-3d316fcb-1ac2-4ea9-ad0f-54ae289355c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:42.354380 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d316fcb-1ac2-4ea9-ad0f-54ae289355c2 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:42.359215 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf20f60-ea5a-4f50-84fa-f68d0f5d3d7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:42.365050 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf20f60-ea5a-4f50-84fa-f68d0f5d3d7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:42.365351 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf20f60-ea5a-4f50-84fa-f68d0f5d3d7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:42.365619 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf20f60-ea5a-4f50-84fa-f68d0f5d3d7d admin admin] PATH_INFO: `/v3/projects/7ec24b38fc844648a4e61575b8ee1c70` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:42.367017 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf20f60-ea5a-4f50-84fa-f68d0f5d3d7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ec24b38fc844648a4e61575b8ee1c70)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:42.369937 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf20f60-ea5a-4f50-84fa-f68d0f5d3d7d admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:42.418118 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-2bf20f60-ea5a-4f50-84fa-f68d0f5d3d7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ec24b38fc844648a4e61575b8ee1c70', 'request_id': 'req-2bf20f60-ea5a-4f50-84fa-f68d0f5d3d7d'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:42.423852 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-2bf20f60-ea5a-4f50-84fa-f68d0f5d3d7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ec24b38fc844648a4e61575b8ee1c70', 'request_id': 'req-2bf20f60-ea5a-4f50-84fa-f68d0f5d3d7d'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:42.429929 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-2bf20f60-ea5a-4f50-84fa-f68d0f5d3d7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ec24b38fc844648a4e61575b8ee1c70', 'request_id': 'req-2bf20f60-ea5a-4f50-84fa-f68d0f5d3d7d'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:42.443917 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-2bf20f60-ea5a-4f50-84fa-f68d0f5d3d7d admin admin] The token cache is being invalidate because project 7ec24b38fc844648a4e61575b8ee1c70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81591) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:13:42.444192 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-2bf20f60-ea5a-4f50-84fa-f68d0f5d3d7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:42.444800 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-2bf20f60-ea5a-4f50-84fa-f68d0f5d3d7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:42.446689 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 978/1935] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 9 12:13:42 2022] DELETE /identity/v3/projects/7ec24b38fc844648a4e61575b8ee1c70 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:13:42.450678 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba85df0-36e0-4905-9cc6-d2fc09804101 service neutron] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:42.460257 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba85df0-36e0-4905-9cc6-d2fc09804101 service neutron] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:42.517506 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca9763d-88cb-4ff9-bdc6-57e872a89edc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:42.518141 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca9763d-88cb-4ff9-bdc6-57e872a89edc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:42.518421 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca9763d-88cb-4ff9-bdc6-57e872a89edc admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:42.518694 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca9763d-88cb-4ff9-bdc6-57e872a89edc admin admin] PATH_INFO: `/v3/users/17570e77a56643178fef6ff53dd83946` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:42.519844 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca9763d-88cb-4ff9-bdc6-57e872a89edc admin admin] RBAC: Authorizing `identity:delete_user(user_id=17570e77a56643178fef6ff53dd83946)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:42.521776 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca9763d-88cb-4ff9-bdc6-57e872a89edc admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:42.569107 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-cca9763d-88cb-4ff9-bdc6-57e872a89edc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17570e77a56643178fef6ff53dd83946'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:42.580965 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-cca9763d-88cb-4ff9-bdc6-57e872a89edc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17570e77a56643178fef6ff53dd83946'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:42.587948 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-cca9763d-88cb-4ff9-bdc6-57e872a89edc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17570e77a56643178fef6ff53dd83946'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:42.595001 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-cca9763d-88cb-4ff9-bdc6-57e872a89edc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17570e77a56643178fef6ff53dd83946'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:42.595611 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-cca9763d-88cb-4ff9-bdc6-57e872a89edc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17570e77a56643178fef6ff53dd83946'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:42.601474 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 958/1936] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 9 12:13:42 2022] DELETE /identity/v3/users/17570e77a56643178fef6ff53dd83946 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:13:42.880035 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf20f60-ea5a-4f50-84fa-f68d0f5d3d7d admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:42.881540 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-2bf20f60-ea5a-4f50-84fa-f68d0f5d3d7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:42.895469 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf20f60-ea5a-4f50-84fa-f68d0f5d3d7d admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:42.953336 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:42.960292 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05d4e4a-6ea5-43dd-a16e-200afe4022e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:42.961071 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05d4e4a-6ea5-43dd-a16e-200afe4022e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:42.961686 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05d4e4a-6ea5-43dd-a16e-200afe4022e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:42.962199 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05d4e4a-6ea5-43dd-a16e-200afe4022e4 admin admin] PATH_INFO: `/v3/projects/5d29cd398a894966b9c482938fad4196` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:42.963886 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05d4e4a-6ea5-43dd-a16e-200afe4022e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d29cd398a894966b9c482938fad4196)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:42.966041 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05d4e4a-6ea5-43dd-a16e-200afe4022e4 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:43.018313 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-c05d4e4a-6ea5-43dd-a16e-200afe4022e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d29cd398a894966b9c482938fad4196', 'request_id': 'req-c05d4e4a-6ea5-43dd-a16e-200afe4022e4'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:43.024135 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-c05d4e4a-6ea5-43dd-a16e-200afe4022e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d29cd398a894966b9c482938fad4196', 'request_id': 'req-c05d4e4a-6ea5-43dd-a16e-200afe4022e4'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:43.030563 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-c05d4e4a-6ea5-43dd-a16e-200afe4022e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d29cd398a894966b9c482938fad4196', 'request_id': 'req-c05d4e4a-6ea5-43dd-a16e-200afe4022e4'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:43.045657 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-c05d4e4a-6ea5-43dd-a16e-200afe4022e4 admin admin] The token cache is being invalidate because project 5d29cd398a894966b9c482938fad4196 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81591) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:13:43.045949 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-c05d4e4a-6ea5-43dd-a16e-200afe4022e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:43.046615 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-c05d4e4a-6ea5-43dd-a16e-200afe4022e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:43.051535 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 979/1937] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 9 12:13:42 2022] DELETE /identity/v3/projects/5d29cd398a894966b9c482938fad4196 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:13:43.053300 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a418674-0e67-4cf2-b43b-696160d77f41 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:43.053764 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a418674-0e67-4cf2-b43b-696160d77f41 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:43.054055 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a418674-0e67-4cf2-b43b-696160d77f41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:43.077584 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-7a418674-0e67-4cf2-b43b-696160d77f41 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:13:43.078836 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-7a418674-0e67-4cf2-b43b-696160d77f41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:43.087648 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:43.160239 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 959/1938] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:13:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:13:43.162420 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63ec515-00b5-4e12-a23a-f583b106f51a None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:43.162789 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63ec515-00b5-4e12-a23a-f583b106f51a None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:43.163023 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63ec515-00b5-4e12-a23a-f583b106f51a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:43.191423 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-d63ec515-00b5-4e12-a23a-f583b106f51a None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:13:43.192846 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-d63ec515-00b5-4e12-a23a-f583b106f51a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:43.206455 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 980/1939] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:13:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:13:43.207132 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a418674-0e67-4cf2-b43b-696160d77f41 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:43.217793 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a418674-0e67-4cf2-b43b-696160d77f41 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:43.222772 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9527a441-c4d5-4a18-bc23-e434e7b18250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:43.223540 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9527a441-c4d5-4a18-bc23-e434e7b18250 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:43.223892 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9527a441-c4d5-4a18-bc23-e434e7b18250 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:43.224223 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9527a441-c4d5-4a18-bc23-e434e7b18250 admin admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:43.224948 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9527a441-c4d5-4a18-bc23-e434e7b18250 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:13:43.225246 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9527a441-c4d5-4a18-bc23-e434e7b18250 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:43.227364 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9527a441-c4d5-4a18-bc23-e434e7b18250 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:43.239790 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 960/1940] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 9 12:13:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:13:43.244972 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63ec515-00b5-4e12-a23a-f583b106f51a None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:43.254946 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63ec515-00b5-4e12-a23a-f583b106f51a None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:43.260149 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad80f0a6-4d88-48cf-89bf-bb2039574df6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:43.260828 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad80f0a6-4d88-48cf-89bf-bb2039574df6 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:43.261167 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad80f0a6-4d88-48cf-89bf-bb2039574df6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:43.261516 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad80f0a6-4d88-48cf-89bf-bb2039574df6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:43.262238 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad80f0a6-4d88-48cf-89bf-bb2039574df6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:43.264243 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad80f0a6-4d88-48cf-89bf-bb2039574df6 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:43.295387 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 981/1941] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 9 12:13:43 2022] POST /identity/v3/projects => generated 357 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:13:43.299740 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9527a441-c4d5-4a18-bc23-e434e7b18250 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:43.311408 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9527a441-c4d5-4a18-bc23-e434e7b18250 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:43.495143 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad80f0a6-4d88-48cf-89bf-bb2039574df6 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:43.497925 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-ad80f0a6-4d88-48cf-89bf-bb2039574df6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:43.512356 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a8ccf5-5307-4ae5-8360-f687c762a693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:43.515568 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a8ccf5-5307-4ae5-8360-f687c762a693 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:43.515919 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a8ccf5-5307-4ae5-8360-f687c762a693 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:43.516197 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a8ccf5-5307-4ae5-8360-f687c762a693 admin admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:43.516971 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a8ccf5-5307-4ae5-8360-f687c762a693 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:43.519457 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a8ccf5-5307-4ae5-8360-f687c762a693 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:43.562968 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 961/1942] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:13:43 2022] POST /identity/v3/users => generated 279 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:13:43.567988 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a8ccf5-5307-4ae5-8360-f687c762a693 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:43.607100 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a8ccf5-5307-4ae5-8360-f687c762a693 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:43.619676 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d904ca-5873-4a81-99c5-cc2d5a28dd2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:43.620301 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d904ca-5873-4a81-99c5-cc2d5a28dd2a admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:43.621819 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d904ca-5873-4a81-99c5-cc2d5a28dd2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:43.622097 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d904ca-5873-4a81-99c5-cc2d5a28dd2a admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:43.622759 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d904ca-5873-4a81-99c5-cc2d5a28dd2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:43.624398 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d904ca-5873-4a81-99c5-cc2d5a28dd2a admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:43.639918 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 962/1943] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:13:43 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:13:43.647506 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d904ca-5873-4a81-99c5-cc2d5a28dd2a admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:43.658546 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad80f0a6-4d88-48cf-89bf-bb2039574df6 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:43.663360 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d904ca-5873-4a81-99c5-cc2d5a28dd2a admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:43.677066 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cd6c18-f5d5-470b-8d73-8f5530e486e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:43.681215 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cd6c18-f5d5-470b-8d73-8f5530e486e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:43.681215 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cd6c18-f5d5-470b-8d73-8f5530e486e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:43.681215 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cd6c18-f5d5-470b-8d73-8f5530e486e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:43.681215 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cd6c18-f5d5-470b-8d73-8f5530e486e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:43.682235 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb7eccf-c51c-46d2-ae7b-8983a71d5270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:43.682926 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb7eccf-c51c-46d2-ae7b-8983a71d5270 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:43.683213 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb7eccf-c51c-46d2-ae7b-8983a71d5270 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:43.683487 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb7eccf-c51c-46d2-ae7b-8983a71d5270 admin admin] PATH_INFO: `/v3/users/f73a124bc3e64e338a7c4542a69c6bb8` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:43.684480 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cd6c18-f5d5-470b-8d73-8f5530e486e3 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:43.690685 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb7eccf-c51c-46d2-ae7b-8983a71d5270 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f73a124bc3e64e338a7c4542a69c6bb8)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:43.690685 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb7eccf-c51c-46d2-ae7b-8983a71d5270 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:43.714369 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 963/1944] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:13:43 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 12:13:43.719330 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cd6c18-f5d5-470b-8d73-8f5530e486e3 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:43.735712 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cd6c18-f5d5-470b-8d73-8f5530e486e3 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:43.742541 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7839f5c-72dc-4fdb-b844-649a1331d351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:43.743231 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7839f5c-72dc-4fdb-b844-649a1331d351 admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:43.743567 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7839f5c-72dc-4fdb-b844-649a1331d351 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:43.743844 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7839f5c-72dc-4fdb-b844-649a1331d351 admin admin] PATH_INFO: `/v3/projects/7e32fba5f14b4c4188a69f37c7613d24/users/2369fa6e679549faa730812dceb1e94f/roles/eecd2f160dd844f5906f3d345780c501` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:43.757525 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-dbb7eccf-c51c-46d2-ae7b-8983a71d5270 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f73a124bc3e64e338a7c4542a69c6bb8'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:43.768820 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-dbb7eccf-c51c-46d2-ae7b-8983a71d5270 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f73a124bc3e64e338a7c4542a69c6bb8'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:43.783830 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7839f5c-72dc-4fdb-b844-649a1331d351 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e32fba5f14b4c4188a69f37c7613d24, user_id=2369fa6e679549faa730812dceb1e94f, role_id=eecd2f160dd844f5906f3d345780c501)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:43.786591 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7839f5c-72dc-4fdb-b844-649a1331d351 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:43.794507 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:43.814221 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-dbb7eccf-c51c-46d2-ae7b-8983a71d5270 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f73a124bc3e64e338a7c4542a69c6bb8'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:43.815782 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 964/1945] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:13:43 2022] PUT /identity/v3/projects/7e32fba5f14b4c4188a69f37c7613d24/users/2369fa6e679549faa730812dceb1e94f/roles/eecd2f160dd844f5906f3d345780c501 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:13:43.828517 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6adc54-3a6b-463b-a1ea-1db4fdd00889 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:43.828813 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6adc54-3a6b-463b-a1ea-1db4fdd00889 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:43.829066 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6adc54-3a6b-463b-a1ea-1db4fdd00889 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:43.830831 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-dbb7eccf-c51c-46d2-ae7b-8983a71d5270 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f73a124bc3e64e338a7c4542a69c6bb8'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:43.832104 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-dbb7eccf-c51c-46d2-ae7b-8983a71d5270 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f73a124bc3e64e338a7c4542a69c6bb8'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:43.840318 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 982/1946] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 9 12:13:43 2022] DELETE /identity/v3/users/f73a124bc3e64e338a7c4542a69c6bb8 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:13:43.846130 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aaa59c-bf4b-4f6a-8022-2045debe10b5 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:43.847125 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aaa59c-bf4b-4f6a-8022-2045debe10b5 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:43.847526 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aaa59c-bf4b-4f6a-8022-2045debe10b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:43.865580 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-7d6adc54-3a6b-463b-a1ea-1db4fdd00889 None None] MFA Rules not processed for user `2369fa6e679549faa730812dceb1e94f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:13:43.866778 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-7d6adc54-3a6b-463b-a1ea-1db4fdd00889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:43.870717 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-93aaa59c-bf4b-4f6a-8022-2045debe10b5 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:13:43.872048 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-93aaa59c-bf4b-4f6a-8022-2045debe10b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:43.913318 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:43.981178 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 965/1947] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:13:43 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:13:43.986444 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6adc54-3a6b-463b-a1ea-1db4fdd00889 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:43.990378 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-7d6adc54-3a6b-463b-a1ea-1db4fdd00889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:43.999412 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 983/1948] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:13:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:13:44.011656 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93aaa59c-bf4b-4f6a-8022-2045debe10b5 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:44.013341 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-93aaa59c-bf4b-4f6a-8022-2045debe10b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:44.026562 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6adc54-3a6b-463b-a1ea-1db4fdd00889 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:44.031871 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca9ca8b-7bb8-4195-bff2-538ca811ce95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:44.032486 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca9ca8b-7bb8-4195-bff2-538ca811ce95 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:44.032767 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca9ca8b-7bb8-4195-bff2-538ca811ce95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:44.033105 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca9ca8b-7bb8-4195-bff2-538ca811ce95 admin admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:44.034042 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca9ca8b-7bb8-4195-bff2-538ca811ce95 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:44.036581 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca9ca8b-7bb8-4195-bff2-538ca811ce95 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:44.085673 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 966/1949] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 9 12:13:43 2022] POST /identity/v3/projects => generated 355 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:13:44.087533 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca9ca8b-7bb8-4195-bff2-538ca811ce95 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:44.100923 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93aaa59c-bf4b-4f6a-8022-2045debe10b5 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:44.206926 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca9ca8b-7bb8-4195-bff2-538ca811ce95 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:44.211706 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec3aa00-5f24-4ee2-a229-2edfe34d3104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:44.212476 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec3aa00-5f24-4ee2-a229-2edfe34d3104 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:44.212756 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec3aa00-5f24-4ee2-a229-2edfe34d3104 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:44.213028 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec3aa00-5f24-4ee2-a229-2edfe34d3104 admin admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:44.213667 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec3aa00-5f24-4ee2-a229-2edfe34d3104 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:44.215653 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec3aa00-5f24-4ee2-a229-2edfe34d3104 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:44.220777 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721ad17d-4f30-40ed-ad9a-a728d7bf889c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:44.221308 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721ad17d-4f30-40ed-ad9a-a728d7bf889c service neutron] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:44.221585 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721ad17d-4f30-40ed-ad9a-a728d7bf889c service neutron] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:44.221852 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721ad17d-4f30-40ed-ad9a-a728d7bf889c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:44.241525 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721ad17d-4f30-40ed-ad9a-a728d7bf889c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:44.246398 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 967/1950] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:13:44 2022] POST /identity/v3/users => generated 284 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:13:44.247779 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721ad17d-4f30-40ed-ad9a-a728d7bf889c service neutron] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:44.247890 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec3aa00-5f24-4ee2-a229-2edfe34d3104 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:44.261000 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec3aa00-5f24-4ee2-a229-2edfe34d3104 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:44.264160 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 984/1951] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:13:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:13:44.265948 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6153646e-e05e-4a31-8832-de973b91b12b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:44.266560 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6153646e-e05e-4a31-8832-de973b91b12b admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:44.266838 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6153646e-e05e-4a31-8832-de973b91b12b admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:44.267136 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6153646e-e05e-4a31-8832-de973b91b12b admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:44.267901 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6153646e-e05e-4a31-8832-de973b91b12b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:44.270348 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6153646e-e05e-4a31-8832-de973b91b12b admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:44.283661 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 968/1952] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:13:44 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 12:13:44.287700 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721ad17d-4f30-40ed-ad9a-a728d7bf889c service neutron] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:44.299414 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721ad17d-4f30-40ed-ad9a-a728d7bf889c service neutron] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:44.304849 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496e8a18-fb39-4963-9b4a-4b4f2184e84e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:44.305570 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496e8a18-fb39-4963-9b4a-4b4f2184e84e admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:44.305856 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496e8a18-fb39-4963-9b4a-4b4f2184e84e admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:44.306131 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496e8a18-fb39-4963-9b4a-4b4f2184e84e admin admin] PATH_INFO: `/v3/projects/f7215d2b1c864f429e926467b5bb0f95/users/a95299c9dd894fe893fa7b13bb99d78c/roles/3f2ca33902a04b6dbc2ef7d06d7289d3` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:44.326179 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496e8a18-fb39-4963-9b4a-4b4f2184e84e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7215d2b1c864f429e926467b5bb0f95, user_id=a95299c9dd894fe893fa7b13bb99d78c, role_id=3f2ca33902a04b6dbc2ef7d06d7289d3)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:44.329379 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496e8a18-fb39-4963-9b4a-4b4f2184e84e admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:44.355151 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 985/1953] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:13:44 2022] PUT /identity/v3/projects/f7215d2b1c864f429e926467b5bb0f95/users/a95299c9dd894fe893fa7b13bb99d78c/roles/3f2ca33902a04b6dbc2ef7d06d7289d3 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:13:44.359190 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6153646e-e05e-4a31-8832-de973b91b12b admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:44.377483 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6153646e-e05e-4a31-8832-de973b91b12b admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:44.450684 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7aaa77-40c7-4c75-82cd-cd70d98259af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:44.451416 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7aaa77-40c7-4c75-82cd-cd70d98259af admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:44.451703 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7aaa77-40c7-4c75-82cd-cd70d98259af admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:44.452027 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7aaa77-40c7-4c75-82cd-cd70d98259af admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:44.455334 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7aaa77-40c7-4c75-82cd-cd70d98259af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:44.455334 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7aaa77-40c7-4c75-82cd-cd70d98259af admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:44.494498 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 969/1954] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:13:44 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:13:44.498527 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496e8a18-fb39-4963-9b4a-4b4f2184e84e admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:44.511779 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496e8a18-fb39-4963-9b4a-4b4f2184e84e admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:44.516923 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cc5a63-f560-4304-aba3-c6a03b06d37d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:44.517671 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cc5a63-f560-4304-aba3-c6a03b06d37d admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:44.518023 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cc5a63-f560-4304-aba3-c6a03b06d37d admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:44.518369 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cc5a63-f560-4304-aba3-c6a03b06d37d admin admin] PATH_INFO: `/v3/projects/f7215d2b1c864f429e926467b5bb0f95/users/a95299c9dd894fe893fa7b13bb99d78c/roles/3f2ca33902a04b6dbc2ef7d06d7289d3` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:44.521076 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cc5a63-f560-4304-aba3-c6a03b06d37d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7215d2b1c864f429e926467b5bb0f95, user_id=a95299c9dd894fe893fa7b13bb99d78c, role_id=3f2ca33902a04b6dbc2ef7d06d7289d3)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:44.525061 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cc5a63-f560-4304-aba3-c6a03b06d37d admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:44.534279 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:44.539960 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 986/1955] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:13:44 2022] PUT /identity/v3/projects/f7215d2b1c864f429e926467b5bb0f95/users/a95299c9dd894fe893fa7b13bb99d78c/roles/3f2ca33902a04b6dbc2ef7d06d7289d3 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:13:44.552517 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7320cd-800d-4a01-ab93-9250f50abad2 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:44.552838 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7320cd-800d-4a01-ab93-9250f50abad2 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:44.553067 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7320cd-800d-4a01-ab93-9250f50abad2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:44.593014 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-3f7320cd-800d-4a01-ab93-9250f50abad2 None None] MFA Rules not processed for user `a95299c9dd894fe893fa7b13bb99d78c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:13:44.594655 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-3f7320cd-800d-4a01-ab93-9250f50abad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:44.619946 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:44.643195 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cc5a63-f560-4304-aba3-c6a03b06d37d admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:44.645135 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-a7cc5a63-f560-4304-aba3-c6a03b06d37d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:44.761583 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 970/1956] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:13:44 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:13:44.777542 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7320cd-800d-4a01-ab93-9250f50abad2 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:44.783292 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-3f7320cd-800d-4a01-ab93-9250f50abad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:44.799245 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cc5a63-f560-4304-aba3-c6a03b06d37d admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:44.806661 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdba4716-732d-4f4b-8e57-1c68dc8f6775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:44.807557 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdba4716-732d-4f4b-8e57-1c68dc8f6775 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:44.808146 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdba4716-732d-4f4b-8e57-1c68dc8f6775 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:44.808638 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdba4716-732d-4f4b-8e57-1c68dc8f6775 admin admin] PATH_INFO: `/v3/projects/f555ce77fce8415888ae4011f760532f` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:44.811620 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdba4716-732d-4f4b-8e57-1c68dc8f6775 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f555ce77fce8415888ae4011f760532f)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:44.814140 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdba4716-732d-4f4b-8e57-1c68dc8f6775 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:44.870276 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7320cd-800d-4a01-ab93-9250f50abad2 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:44.887667 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-fdba4716-732d-4f4b-8e57-1c68dc8f6775 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f555ce77fce8415888ae4011f760532f', 'request_id': 'req-fdba4716-732d-4f4b-8e57-1c68dc8f6775'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:44.906250 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-fdba4716-732d-4f4b-8e57-1c68dc8f6775 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f555ce77fce8415888ae4011f760532f', 'request_id': 'req-fdba4716-732d-4f4b-8e57-1c68dc8f6775'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:44.922438 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-fdba4716-732d-4f4b-8e57-1c68dc8f6775 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f555ce77fce8415888ae4011f760532f', 'request_id': 'req-fdba4716-732d-4f4b-8e57-1c68dc8f6775'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:44.945437 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-fdba4716-732d-4f4b-8e57-1c68dc8f6775 admin admin] The token cache is being invalidate because project f555ce77fce8415888ae4011f760532f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81591) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:13:44.946940 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-fdba4716-732d-4f4b-8e57-1c68dc8f6775 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:44.948752 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-fdba4716-732d-4f4b-8e57-1c68dc8f6775 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:44.951870 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 987/1957] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 9 12:13:44 2022] DELETE /identity/v3/projects/f555ce77fce8415888ae4011f760532f => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:13:45.015615 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa4cf54-7c4a-443a-b84b-250791943615 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535c258100df40b58a5ee44b2c21d621', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:45.016337 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa4cf54-7c4a-443a-b84b-250791943615 service cinder] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:45.016626 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa4cf54-7c4a-443a-b84b-250791943615 service cinder] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:45.016899 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa4cf54-7c4a-443a-b84b-250791943615 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:45.019221 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-2aa4cf54-7c4a-443a-b84b-250791943615 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:45.111683 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa4cf54-7c4a-443a-b84b-250791943615 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:45.113476 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa4cf54-7c4a-443a-b84b-250791943615 service cinder] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:45.126892 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 971/1958] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:13:44 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:13:45.351303 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdba4716-732d-4f4b-8e57-1c68dc8f6775 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:45.351303 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-fdba4716-732d-4f4b-8e57-1c68dc8f6775 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:45.365786 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdba4716-732d-4f4b-8e57-1c68dc8f6775 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:45.383872 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3441cfdd-71c5-40c0-a26e-caafebb7a607 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535c258100df40b58a5ee44b2c21d621', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:45.383872 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3441cfdd-71c5-40c0-a26e-caafebb7a607 service cinder] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:45.383872 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3441cfdd-71c5-40c0-a26e-caafebb7a607 service cinder] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:45.383872 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3441cfdd-71c5-40c0-a26e-caafebb7a607 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:45.384469 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-3441cfdd-71c5-40c0-a26e-caafebb7a607 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:45.526109 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3441cfdd-71c5-40c0-a26e-caafebb7a607 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:45.528018 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3441cfdd-71c5-40c0-a26e-caafebb7a607 service cinder] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:45.550847 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 988/1959] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:13:45 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:13:48.144951 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa4cf54-7c4a-443a-b84b-250791943615 service cinder] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:48.144951 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-2aa4cf54-7c4a-443a-b84b-250791943615 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:48.169333 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:48.237378 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa4cf54-7c4a-443a-b84b-250791943615 service cinder] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:48.243840 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8773a7e-2e8f-457a-941a-52665e2edcae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:48.244571 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8773a7e-2e8f-457a-941a-52665e2edcae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:48.244930 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8773a7e-2e8f-457a-941a-52665e2edcae admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:48.245332 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8773a7e-2e8f-457a-941a-52665e2edcae admin admin] PATH_INFO: `/v3/users/3503d4622e66454e8c81d718072d1a60` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:48.246635 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8773a7e-2e8f-457a-941a-52665e2edcae admin admin] RBAC: Authorizing `identity:delete_user(user_id=3503d4622e66454e8c81d718072d1a60)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:48.249636 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8773a7e-2e8f-457a-941a-52665e2edcae admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:48.308555 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-f8773a7e-2e8f-457a-941a-52665e2edcae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3503d4622e66454e8c81d718072d1a60'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:48.324314 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-f8773a7e-2e8f-457a-941a-52665e2edcae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3503d4622e66454e8c81d718072d1a60'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:48.341123 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-f8773a7e-2e8f-457a-941a-52665e2edcae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3503d4622e66454e8c81d718072d1a60'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:48.355096 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-f8773a7e-2e8f-457a-941a-52665e2edcae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3503d4622e66454e8c81d718072d1a60'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:48.356032 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-f8773a7e-2e8f-457a-941a-52665e2edcae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3503d4622e66454e8c81d718072d1a60'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:48.364590 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 972/1960] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 9 12:13:48 2022] DELETE /identity/v3/users/3503d4622e66454e8c81d718072d1a60 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:13:48.369352 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7d1fad-f964-4dd4-b025-c7cf50bf35b1 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:48.369875 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7d1fad-f964-4dd4-b025-c7cf50bf35b1 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:48.369971 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7d1fad-f964-4dd4-b025-c7cf50bf35b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:48.403085 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-0c7d1fad-f964-4dd4-b025-c7cf50bf35b1 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:13:48.403085 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-0c7d1fad-f964-4dd4-b025-c7cf50bf35b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:48.444336 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:48.516126 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 989/1961] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:13:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:13:48.530133 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8773a7e-2e8f-457a-941a-52665e2edcae admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:48.531618 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-f8773a7e-2e8f-457a-941a-52665e2edcae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:48.616688 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8773a7e-2e8f-457a-941a-52665e2edcae admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:48.621341 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b292c7-45d8-4471-9b86-725e0030716b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:48.621996 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b292c7-45d8-4471-9b86-725e0030716b service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:48.622280 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b292c7-45d8-4471-9b86-725e0030716b service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:48.622552 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b292c7-45d8-4471-9b86-725e0030716b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:48.636204 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b292c7-45d8-4471-9b86-725e0030716b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:48.637705 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b292c7-45d8-4471-9b86-725e0030716b service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:48.653718 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 973/1962] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:13:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:13:48.990974 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7d1fad-f964-4dd4-b025-c7cf50bf35b1 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:48.993346 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-0c7d1fad-f964-4dd4-b025-c7cf50bf35b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:49.007597 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7d1fad-f964-4dd4-b025-c7cf50bf35b1 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:49.014951 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce9aea6-16a3-4703-8483-cad3f0d331e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:49.019078 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce9aea6-16a3-4703-8483-cad3f0d331e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:49.019078 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce9aea6-16a3-4703-8483-cad3f0d331e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:49.019078 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce9aea6-16a3-4703-8483-cad3f0d331e7 admin admin] PATH_INFO: `/v3/projects/f316f1990c07480fa7258787b1c2c47f` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:49.025118 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce9aea6-16a3-4703-8483-cad3f0d331e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f316f1990c07480fa7258787b1c2c47f)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:49.027147 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce9aea6-16a3-4703-8483-cad3f0d331e7 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:49.091839 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-cce9aea6-16a3-4703-8483-cad3f0d331e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f316f1990c07480fa7258787b1c2c47f', 'request_id': 'req-cce9aea6-16a3-4703-8483-cad3f0d331e7'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:49.100907 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-cce9aea6-16a3-4703-8483-cad3f0d331e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f316f1990c07480fa7258787b1c2c47f', 'request_id': 'req-cce9aea6-16a3-4703-8483-cad3f0d331e7'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:49.109183 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-cce9aea6-16a3-4703-8483-cad3f0d331e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f316f1990c07480fa7258787b1c2c47f', 'request_id': 'req-cce9aea6-16a3-4703-8483-cad3f0d331e7'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:49.133364 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-cce9aea6-16a3-4703-8483-cad3f0d331e7 admin admin] The token cache is being invalidate because project f316f1990c07480fa7258787b1c2c47f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81591) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:13:49.133802 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-cce9aea6-16a3-4703-8483-cad3f0d331e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:49.134485 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-cce9aea6-16a3-4703-8483-cad3f0d331e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:49.137149 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 990/1963] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 9 12:13:48 2022] DELETE /identity/v3/projects/f316f1990c07480fa7258787b1c2c47f => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:13:49.141937 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfd0799-e88a-4eda-b1c4-6c19bd54c907 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:49.142261 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfd0799-e88a-4eda-b1c4-6c19bd54c907 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:49.144202 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfd0799-e88a-4eda-b1c4-6c19bd54c907 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:49.172518 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-1dfd0799-e88a-4eda-b1c4-6c19bd54c907 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:13:49.173748 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-1dfd0799-e88a-4eda-b1c4-6c19bd54c907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:49.213109 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:49.252813 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 974/1964] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:13:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:13:49.254762 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030ecff2-ceec-47f9-b4e7-634c55dc01cc None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:49.255082 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030ecff2-ceec-47f9-b4e7-634c55dc01cc None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:49.255354 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030ecff2-ceec-47f9-b4e7-634c55dc01cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:49.276574 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-030ecff2-ceec-47f9-b4e7-634c55dc01cc None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:13:49.277712 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-030ecff2-ceec-47f9-b4e7-634c55dc01cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:49.291675 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 991/1965] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:13:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:13:49.295945 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfd0799-e88a-4eda-b1c4-6c19bd54c907 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:49.307373 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfd0799-e88a-4eda-b1c4-6c19bd54c907 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:49.314388 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a631c5-28bd-443f-b988-c959c2ec360e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:49.315049 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a631c5-28bd-443f-b988-c959c2ec360e admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:49.315307 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a631c5-28bd-443f-b988-c959c2ec360e admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:49.315574 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a631c5-28bd-443f-b988-c959c2ec360e admin admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:49.316221 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a631c5-28bd-443f-b988-c959c2ec360e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:13:49.316519 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a631c5-28bd-443f-b988-c959c2ec360e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:49.318102 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a631c5-28bd-443f-b988-c959c2ec360e admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:49.330890 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 975/1966] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 9 12:13:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:13:49.335729 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030ecff2-ceec-47f9-b4e7-634c55dc01cc None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:49.345650 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030ecff2-ceec-47f9-b4e7-634c55dc01cc None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:49.350265 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ade8030-ff0f-4892-a06b-8a2bbb40215e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:49.350872 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ade8030-ff0f-4892-a06b-8a2bbb40215e admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:49.351194 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ade8030-ff0f-4892-a06b-8a2bbb40215e admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:49.352238 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ade8030-ff0f-4892-a06b-8a2bbb40215e admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:49.352874 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ade8030-ff0f-4892-a06b-8a2bbb40215e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:49.354809 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ade8030-ff0f-4892-a06b-8a2bbb40215e admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:49.384436 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 992/1967] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 9 12:13:49 2022] POST /identity/v3/projects => generated 375 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:13:49.388653 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a631c5-28bd-443f-b988-c959c2ec360e admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:49.401188 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a631c5-28bd-443f-b988-c959c2ec360e admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:49.464942 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaac9a7-ba7d-4b06-b64b-f9121072e62b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:49.465633 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaac9a7-ba7d-4b06-b64b-f9121072e62b admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:49.466019 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaac9a7-ba7d-4b06-b64b-f9121072e62b admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:49.466292 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaac9a7-ba7d-4b06-b64b-f9121072e62b admin admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:49.466932 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaac9a7-ba7d-4b06-b64b-f9121072e62b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:49.468965 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaac9a7-ba7d-4b06-b64b-f9121072e62b admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:49.494817 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 976/1968] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:13:49 2022] POST /identity/v3/users => generated 288 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:13:49.496330 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ade8030-ff0f-4892-a06b-8a2bbb40215e admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:49.508763 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ade8030-ff0f-4892-a06b-8a2bbb40215e admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:49.513590 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c10529e-dedc-43e6-90ba-8e9e80212c02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:49.514257 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c10529e-dedc-43e6-90ba-8e9e80212c02 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:49.514585 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c10529e-dedc-43e6-90ba-8e9e80212c02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:49.514861 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c10529e-dedc-43e6-90ba-8e9e80212c02 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:49.515564 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c10529e-dedc-43e6-90ba-8e9e80212c02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:49.517217 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c10529e-dedc-43e6-90ba-8e9e80212c02 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:49.533109 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 993/1969] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:13:49 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:13:49.534275 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaac9a7-ba7d-4b06-b64b-f9121072e62b admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:49.551722 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaac9a7-ba7d-4b06-b64b-f9121072e62b admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:49.557729 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f139d9c-614d-4654-9520-e77688c76612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:49.558350 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f139d9c-614d-4654-9520-e77688c76612 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:49.558630 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f139d9c-614d-4654-9520-e77688c76612 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:49.559669 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f139d9c-614d-4654-9520-e77688c76612 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:49.560642 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f139d9c-614d-4654-9520-e77688c76612 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:49.562410 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f139d9c-614d-4654-9520-e77688c76612 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:49.580101 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 977/1970] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:13:49 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:13:49.581253 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c10529e-dedc-43e6-90ba-8e9e80212c02 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:49.592927 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c10529e-dedc-43e6-90ba-8e9e80212c02 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:49.600519 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0040bea-8309-43bd-947a-ab269b4adaaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:49.601115 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0040bea-8309-43bd-947a-ab269b4adaaa admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:49.601399 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0040bea-8309-43bd-947a-ab269b4adaaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:49.602395 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0040bea-8309-43bd-947a-ab269b4adaaa admin admin] PATH_INFO: `/v3/projects/14a38b41bf08445c9f1995c27de797fa/users/7c4c826495c24b00ace50e88e4725432/roles/eecd2f160dd844f5906f3d345780c501` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:49.646125 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0040bea-8309-43bd-947a-ab269b4adaaa admin admin] RBAC: Authorizing `identity:create_grant(project_id=14a38b41bf08445c9f1995c27de797fa, user_id=7c4c826495c24b00ace50e88e4725432, role_id=eecd2f160dd844f5906f3d345780c501)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:49.646125 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0040bea-8309-43bd-947a-ab269b4adaaa admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:49.678061 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 994/1971] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:13:49 2022] PUT /identity/v3/projects/14a38b41bf08445c9f1995c27de797fa/users/7c4c826495c24b00ace50e88e4725432/roles/eecd2f160dd844f5906f3d345780c501 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:13:49.693629 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517e39cd-610b-4225-88db-62760c23b8ab None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:49.693629 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517e39cd-610b-4225-88db-62760c23b8ab None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:49.693629 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517e39cd-610b-4225-88db-62760c23b8ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:49.734557 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-517e39cd-610b-4225-88db-62760c23b8ab None None] MFA Rules not processed for user `7c4c826495c24b00ace50e88e4725432`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:13:49.735814 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-517e39cd-610b-4225-88db-62760c23b8ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:49.821852 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 978/1972] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:13:49 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:13:49.832560 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0040bea-8309-43bd-947a-ab269b4adaaa admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:49.834417 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-f0040bea-8309-43bd-947a-ab269b4adaaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:49.896183 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:49.901933 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0040bea-8309-43bd-947a-ab269b4adaaa admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:49.908017 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ec74c6-a182-4c70-93d0-003d313ae9d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535c258100df40b58a5ee44b2c21d621', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:49.909228 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ec74c6-a182-4c70-93d0-003d313ae9d1 service cinder] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:49.909817 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ec74c6-a182-4c70-93d0-003d313ae9d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:49.910660 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ec74c6-a182-4c70-93d0-003d313ae9d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:49.926029 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ec74c6-a182-4c70-93d0-003d313ae9d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:49.927823 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ec74c6-a182-4c70-93d0-003d313ae9d1 service cinder] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:49.956954 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 995/1973] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:13:49 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:13:53.313586 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517e39cd-610b-4225-88db-62760c23b8ab None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:53.367651 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:53.394572 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517e39cd-610b-4225-88db-62760c23b8ab None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:53.400196 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638932b6-df98-4eda-a27d-1b0e8d8f2307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:53.400834 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638932b6-df98-4eda-a27d-1b0e8d8f2307 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:53.401110 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638932b6-df98-4eda-a27d-1b0e8d8f2307 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:53.402105 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638932b6-df98-4eda-a27d-1b0e8d8f2307 admin admin] PATH_INFO: `/v3/users/7c4c826495c24b00ace50e88e4725432` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:53.402776 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638932b6-df98-4eda-a27d-1b0e8d8f2307 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c4c826495c24b00ace50e88e4725432)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:53.404857 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638932b6-df98-4eda-a27d-1b0e8d8f2307 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:53.450217 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-638932b6-df98-4eda-a27d-1b0e8d8f2307 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c4c826495c24b00ace50e88e4725432'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:53.462344 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-638932b6-df98-4eda-a27d-1b0e8d8f2307 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c4c826495c24b00ace50e88e4725432'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:53.469119 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-638932b6-df98-4eda-a27d-1b0e8d8f2307 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c4c826495c24b00ace50e88e4725432'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:53.477521 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-638932b6-df98-4eda-a27d-1b0e8d8f2307 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c4c826495c24b00ace50e88e4725432'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:53.478295 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-638932b6-df98-4eda-a27d-1b0e8d8f2307 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c4c826495c24b00ace50e88e4725432'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:53.484385 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 979/1974] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 9 12:13:53 2022] DELETE /identity/v3/users/7c4c826495c24b00ace50e88e4725432 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:13:53.489057 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd83d9f-4890-4188-b17c-fa27f3d2b892 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:53.489534 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd83d9f-4890-4188-b17c-fa27f3d2b892 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:53.489671 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd83d9f-4890-4188-b17c-fa27f3d2b892 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:53.510096 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-3fd83d9f-4890-4188-b17c-fa27f3d2b892 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:13:53.511357 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-3fd83d9f-4890-4188-b17c-fa27f3d2b892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:53.571513 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 996/1975] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:13:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:13:53.581172 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638932b6-df98-4eda-a27d-1b0e8d8f2307 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:53.582572 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-638932b6-df98-4eda-a27d-1b0e8d8f2307 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:53.639764 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638932b6-df98-4eda-a27d-1b0e8d8f2307 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:53.643402 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9ea9f2-1ca9-40f9-b103-b1200c91810c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:53.644011 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9ea9f2-1ca9-40f9-b103-b1200c91810c service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:53.644297 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9ea9f2-1ca9-40f9-b103-b1200c91810c service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:53.644566 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9ea9f2-1ca9-40f9-b103-b1200c91810c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:53.655840 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9ea9f2-1ca9-40f9-b103-b1200c91810c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:53.657278 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9ea9f2-1ca9-40f9-b103-b1200c91810c service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:53.668584 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 980/1976] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:13:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:13:53.940425 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd83d9f-4890-4188-b17c-fa27f3d2b892 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:53.941898 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-3fd83d9f-4890-4188-b17c-fa27f3d2b892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:53.954801 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd83d9f-4890-4188-b17c-fa27f3d2b892 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:53.959669 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b554e9-6770-4e60-83ae-f5251fcc259c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:53.960310 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b554e9-6770-4e60-83ae-f5251fcc259c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:53.960587 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b554e9-6770-4e60-83ae-f5251fcc259c admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:53.961050 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b554e9-6770-4e60-83ae-f5251fcc259c admin admin] PATH_INFO: `/v3/projects/14a38b41bf08445c9f1995c27de797fa` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:53.963332 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b554e9-6770-4e60-83ae-f5251fcc259c admin admin] RBAC: Authorizing `identity:delete_project(project_id=14a38b41bf08445c9f1995c27de797fa)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:53.965551 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b554e9-6770-4e60-83ae-f5251fcc259c admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:54.016228 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-b1b554e9-6770-4e60-83ae-f5251fcc259c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14a38b41bf08445c9f1995c27de797fa', 'request_id': 'req-b1b554e9-6770-4e60-83ae-f5251fcc259c'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:54.021932 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-b1b554e9-6770-4e60-83ae-f5251fcc259c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14a38b41bf08445c9f1995c27de797fa', 'request_id': 'req-b1b554e9-6770-4e60-83ae-f5251fcc259c'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:54.028452 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-b1b554e9-6770-4e60-83ae-f5251fcc259c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14a38b41bf08445c9f1995c27de797fa', 'request_id': 'req-b1b554e9-6770-4e60-83ae-f5251fcc259c'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:54.040556 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-b1b554e9-6770-4e60-83ae-f5251fcc259c admin admin] The token cache is being invalidate because project 14a38b41bf08445c9f1995c27de797fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81591) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:13:54.040855 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-b1b554e9-6770-4e60-83ae-f5251fcc259c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:54.041467 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-b1b554e9-6770-4e60-83ae-f5251fcc259c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:13:54.042971 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 997/1977] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 9 12:13:53 2022] DELETE /identity/v3/projects/14a38b41bf08445c9f1995c27de797fa => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:13:54.047806 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead151ea-2108-4e70-ad1b-c31ebd799555 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:54.048120 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead151ea-2108-4e70-ad1b-c31ebd799555 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:54.048395 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead151ea-2108-4e70-ad1b-c31ebd799555 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:54.068619 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-ead151ea-2108-4e70-ad1b-c31ebd799555 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:13:54.069813 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-ead151ea-2108-4e70-ad1b-c31ebd799555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:54.128420 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:54.140873 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 981/1978] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:13:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:13:54.149984 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab690ab7-640b-409c-82f7-59b585c8c0dc None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:54.150288 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab690ab7-640b-409c-82f7-59b585c8c0dc None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:54.150541 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab690ab7-640b-409c-82f7-59b585c8c0dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:54.172126 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-ab690ab7-640b-409c-82f7-59b585c8c0dc None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:13:54.173338 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-ab690ab7-640b-409c-82f7-59b585c8c0dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:54.182350 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 998/1979] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:13:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:13:54.187035 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead151ea-2108-4e70-ad1b-c31ebd799555 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:54.197181 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead151ea-2108-4e70-ad1b-c31ebd799555 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:54.202164 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78ccda0-311d-4c3d-ab09-c49a9f3ac51c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:54.202771 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78ccda0-311d-4c3d-ab09-c49a9f3ac51c admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:54.203078 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78ccda0-311d-4c3d-ab09-c49a9f3ac51c admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:54.203348 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78ccda0-311d-4c3d-ab09-c49a9f3ac51c admin admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:54.203988 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78ccda0-311d-4c3d-ab09-c49a9f3ac51c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:13:54.204284 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78ccda0-311d-4c3d-ab09-c49a9f3ac51c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:54.205867 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78ccda0-311d-4c3d-ab09-c49a9f3ac51c admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:54.238230 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 982/1980] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 9 12:13:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:13:54.243767 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab690ab7-640b-409c-82f7-59b585c8c0dc None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:54.253203 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab690ab7-640b-409c-82f7-59b585c8c0dc None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:54.258214 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42528008-3a40-4106-b20a-adec1d125754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:54.259699 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42528008-3a40-4106-b20a-adec1d125754 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:54.259978 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42528008-3a40-4106-b20a-adec1d125754 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:54.260253 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42528008-3a40-4106-b20a-adec1d125754 admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:54.260886 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42528008-3a40-4106-b20a-adec1d125754 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:54.262770 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42528008-3a40-4106-b20a-adec1d125754 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:54.295835 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 999/1981] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 9 12:13:54 2022] POST /identity/v3/projects => generated 371 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:13:54.298654 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78ccda0-311d-4c3d-ab09-c49a9f3ac51c admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:54.313818 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78ccda0-311d-4c3d-ab09-c49a9f3ac51c admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:54.390932 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1c6845-f683-496d-9f5f-cec75722f546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:54.391655 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1c6845-f683-496d-9f5f-cec75722f546 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:54.392002 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1c6845-f683-496d-9f5f-cec75722f546 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:54.392365 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1c6845-f683-496d-9f5f-cec75722f546 admin admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:54.393035 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1c6845-f683-496d-9f5f-cec75722f546 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:54.395057 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1c6845-f683-496d-9f5f-cec75722f546 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:54.421426 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 983/1982] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:13:54 2022] POST /identity/v3/users => generated 286 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:13:54.423574 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42528008-3a40-4106-b20a-adec1d125754 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:54.436071 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42528008-3a40-4106-b20a-adec1d125754 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:54.447903 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2a7196-d214-4d26-a6e5-a25f3f81a392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:54.448752 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2a7196-d214-4d26-a6e5-a25f3f81a392 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:54.449101 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2a7196-d214-4d26-a6e5-a25f3f81a392 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:54.449437 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2a7196-d214-4d26-a6e5-a25f3f81a392 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:54.450159 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2a7196-d214-4d26-a6e5-a25f3f81a392 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:54.452153 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2a7196-d214-4d26-a6e5-a25f3f81a392 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:54.464796 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1000/1983] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:13:54 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:13:54.468795 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1c6845-f683-496d-9f5f-cec75722f546 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:54.483287 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1c6845-f683-496d-9f5f-cec75722f546 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:54.489594 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cee8ba-af63-449f-8ade-c30119eb6450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:54.490234 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cee8ba-af63-449f-8ade-c30119eb6450 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:54.490509 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cee8ba-af63-449f-8ade-c30119eb6450 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:54.490951 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cee8ba-af63-449f-8ade-c30119eb6450 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:54.491576 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cee8ba-af63-449f-8ade-c30119eb6450 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:54.494528 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cee8ba-af63-449f-8ade-c30119eb6450 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:54.507537 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 984/1984] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:13:54 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:13:54.512203 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2a7196-d214-4d26-a6e5-a25f3f81a392 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:54.524182 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2a7196-d214-4d26-a6e5-a25f3f81a392 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:54.529167 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0580113b-d992-42c0-b4af-450736007e01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:54.529885 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0580113b-d992-42c0-b4af-450736007e01 admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:54.530075 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0580113b-d992-42c0-b4af-450736007e01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:54.530310 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0580113b-d992-42c0-b4af-450736007e01 admin admin] PATH_INFO: `/v3/projects/80d60a0051c24f9b98bc252a0d0754cc/users/913344053b6244c584140807ab2149cb/roles/eecd2f160dd844f5906f3d345780c501` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:54.548897 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0580113b-d992-42c0-b4af-450736007e01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80d60a0051c24f9b98bc252a0d0754cc, user_id=913344053b6244c584140807ab2149cb, role_id=eecd2f160dd844f5906f3d345780c501)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:54.551524 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0580113b-d992-42c0-b4af-450736007e01 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:54.566096 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1001/1985] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:13:54 2022] PUT /identity/v3/projects/80d60a0051c24f9b98bc252a0d0754cc/users/913344053b6244c584140807ab2149cb/roles/eecd2f160dd844f5906f3d345780c501 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:13:54.571471 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a2b2c7-359a-4d9e-abd3-f20ff00a2a4a None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:54.571754 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a2b2c7-359a-4d9e-abd3-f20ff00a2a4a None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:54.572052 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a2b2c7-359a-4d9e-abd3-f20ff00a2a4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:54.590358 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-42a2b2c7-359a-4d9e-abd3-f20ff00a2a4a None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:13:54.591576 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-42a2b2c7-359a-4d9e-abd3-f20ff00a2a4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:54.652373 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 985/1986] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:13:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:13:54.661996 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0580113b-d992-42c0-b4af-450736007e01 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:54.663703 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-0580113b-d992-42c0-b4af-450736007e01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:54.677897 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:54.729766 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0580113b-d992-42c0-b4af-450736007e01 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:54.733383 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f42072d-5057-4a41-84fe-34c747570af6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:54.733988 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f42072d-5057-4a41-84fe-34c747570af6 service neutron] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:54.734264 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f42072d-5057-4a41-84fe-34c747570af6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:54.734538 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f42072d-5057-4a41-84fe-34c747570af6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:54.744072 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f42072d-5057-4a41-84fe-34c747570af6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:54.745509 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f42072d-5057-4a41-84fe-34c747570af6 service neutron] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:54.755212 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1002/1987] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:13:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:13:55.412860 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567e2ad8-dda4-4a15-87d2-48e5f25f8ec6 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:55.413295 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567e2ad8-dda4-4a15-87d2-48e5f25f8ec6 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:55.413409 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567e2ad8-dda4-4a15-87d2-48e5f25f8ec6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:55.445167 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-567e2ad8-dda4-4a15-87d2-48e5f25f8ec6 None None] MFA Rules not processed for user `913344053b6244c584140807ab2149cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:13:55.446401 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-567e2ad8-dda4-4a15-87d2-48e5f25f8ec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:55.514778 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:13:55.537286 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 986/1988] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:13:55 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:13:55.547316 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f42072d-5057-4a41-84fe-34c747570af6 service neutron] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:13:55.548720 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-8f42072d-5057-4a41-84fe-34c747570af6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:55.645285 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f42072d-5057-4a41-84fe-34c747570af6 service neutron] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:13:55.650036 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40a26ec-8bb7-42cc-a4ea-8ec76e8114f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535c258100df40b58a5ee44b2c21d621', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:13:55.650952 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40a26ec-8bb7-42cc-a4ea-8ec76e8114f1 service cinder] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:55.651240 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40a26ec-8bb7-42cc-a4ea-8ec76e8114f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:55.651506 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40a26ec-8bb7-42cc-a4ea-8ec76e8114f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:55.660401 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40a26ec-8bb7-42cc-a4ea-8ec76e8114f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:13:55.661847 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40a26ec-8bb7-42cc-a4ea-8ec76e8114f1 service cinder] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:13:55.672383 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1003/1989] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:13:55 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:13:57.094128 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c725b78c-49f2-4ef1-9994-04f7818d689c None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:13:57.094979 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c725b78c-49f2-4ef1-9994-04f7818d689c None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:13:57.095467 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c725b78c-49f2-4ef1-9994-04f7818d689c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:13:57.131794 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-c725b78c-49f2-4ef1-9994-04f7818d689c None None] MFA Rules not processed for user `913344053b6244c584140807ab2149cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:13:57.134273 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-c725b78c-49f2-4ef1-9994-04f7818d689c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:13:57.147724 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 987/1990] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:13:57 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:14:01.372208 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40a26ec-8bb7-42cc-a4ea-8ec76e8114f1 service cinder] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:01.374549 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-b40a26ec-8bb7-42cc-a4ea-8ec76e8114f1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:01.403422 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40a26ec-8bb7-42cc-a4ea-8ec76e8114f1 service cinder] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:01.412332 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b498ad47-ac45-4ffd-959c-6c96b75230d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:01.413378 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b498ad47-ac45-4ffd-959c-6c96b75230d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:01.413894 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b498ad47-ac45-4ffd-959c-6c96b75230d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:01.414399 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b498ad47-ac45-4ffd-959c-6c96b75230d7 admin admin] PATH_INFO: `/v3/users/2369fa6e679549faa730812dceb1e94f` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:01.417556 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b498ad47-ac45-4ffd-959c-6c96b75230d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2369fa6e679549faa730812dceb1e94f)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:01.421495 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b498ad47-ac45-4ffd-959c-6c96b75230d7 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:01.512073 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-b498ad47-ac45-4ffd-959c-6c96b75230d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2369fa6e679549faa730812dceb1e94f'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:01.524170 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-b498ad47-ac45-4ffd-959c-6c96b75230d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2369fa6e679549faa730812dceb1e94f'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:01.550378 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-b498ad47-ac45-4ffd-959c-6c96b75230d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2369fa6e679549faa730812dceb1e94f'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:01.562828 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-b498ad47-ac45-4ffd-959c-6c96b75230d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2369fa6e679549faa730812dceb1e94f'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:01.563720 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-b498ad47-ac45-4ffd-959c-6c96b75230d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2369fa6e679549faa730812dceb1e94f'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:01.573840 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1004/1991] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 9 12:14:01 2022] DELETE /identity/v3/users/2369fa6e679549faa730812dceb1e94f => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:14:01.575397 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c408479e-bbb3-445b-b92a-5cd94197ca0e None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:01.575865 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c408479e-bbb3-445b-b92a-5cd94197ca0e None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:01.575956 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c408479e-bbb3-445b-b92a-5cd94197ca0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:01.596462 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-c408479e-bbb3-445b-b92a-5cd94197ca0e None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:14:01.597982 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-c408479e-bbb3-445b-b92a-5cd94197ca0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:01.675841 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 988/1992] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:14:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:14:01.688208 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b498ad47-ac45-4ffd-959c-6c96b75230d7 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:01.690150 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-b498ad47-ac45-4ffd-959c-6c96b75230d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:01.727048 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:14:01.761399 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b498ad47-ac45-4ffd-959c-6c96b75230d7 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:01.767014 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1797c553-134b-4875-9fea-69f9134fbc64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:01.767879 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1797c553-134b-4875-9fea-69f9134fbc64 service neutron] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:01.768654 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1797c553-134b-4875-9fea-69f9134fbc64 service neutron] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:01.769577 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1797c553-134b-4875-9fea-69f9134fbc64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:01.781139 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1797c553-134b-4875-9fea-69f9134fbc64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:01.782667 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1797c553-134b-4875-9fea-69f9134fbc64 service neutron] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:01.794562 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1005/1993] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:14:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:14:02.232594 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c408479e-bbb3-445b-b92a-5cd94197ca0e None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:02.234661 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-c408479e-bbb3-445b-b92a-5cd94197ca0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:02.247593 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c408479e-bbb3-445b-b92a-5cd94197ca0e None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:02.252848 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a62e80-c176-4c9e-bf30-0e6aff8b93c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:02.253562 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a62e80-c176-4c9e-bf30-0e6aff8b93c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:02.255495 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a62e80-c176-4c9e-bf30-0e6aff8b93c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:02.255777 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a62e80-c176-4c9e-bf30-0e6aff8b93c1 admin admin] PATH_INFO: `/v3/projects/7e32fba5f14b4c4188a69f37c7613d24` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:02.260724 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a62e80-c176-4c9e-bf30-0e6aff8b93c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e32fba5f14b4c4188a69f37c7613d24)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:02.262702 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a62e80-c176-4c9e-bf30-0e6aff8b93c1 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:02.333365 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-f5a62e80-c176-4c9e-bf30-0e6aff8b93c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e32fba5f14b4c4188a69f37c7613d24', 'request_id': 'req-f5a62e80-c176-4c9e-bf30-0e6aff8b93c1'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:02.344180 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-f5a62e80-c176-4c9e-bf30-0e6aff8b93c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e32fba5f14b4c4188a69f37c7613d24', 'request_id': 'req-f5a62e80-c176-4c9e-bf30-0e6aff8b93c1'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:02.357908 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-f5a62e80-c176-4c9e-bf30-0e6aff8b93c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e32fba5f14b4c4188a69f37c7613d24', 'request_id': 'req-f5a62e80-c176-4c9e-bf30-0e6aff8b93c1'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:02.391213 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-f5a62e80-c176-4c9e-bf30-0e6aff8b93c1 admin admin] The token cache is being invalidate because project 7e32fba5f14b4c4188a69f37c7613d24 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:14:02.391576 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-f5a62e80-c176-4c9e-bf30-0e6aff8b93c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:02.392475 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-f5a62e80-c176-4c9e-bf30-0e6aff8b93c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:02.394040 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 989/1994] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 9 12:14:02 2022] DELETE /identity/v3/projects/7e32fba5f14b4c4188a69f37c7613d24 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:14:02.398129 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1797c553-134b-4875-9fea-69f9134fbc64 service neutron] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:02.408323 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1797c553-134b-4875-9fea-69f9134fbc64 service neutron] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:02.484637 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c14f87f-9413-43fb-820e-b7a6fb7db210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:02.484939 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c14f87f-9413-43fb-820e-b7a6fb7db210 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:02.485250 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c14f87f-9413-43fb-820e-b7a6fb7db210 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:02.485523 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c14f87f-9413-43fb-820e-b7a6fb7db210 admin admin] PATH_INFO: `/v3/users/a95299c9dd894fe893fa7b13bb99d78c` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:02.487049 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c14f87f-9413-43fb-820e-b7a6fb7db210 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a95299c9dd894fe893fa7b13bb99d78c)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:02.489055 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c14f87f-9413-43fb-820e-b7a6fb7db210 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:02.555256 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-6c14f87f-9413-43fb-820e-b7a6fb7db210 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a95299c9dd894fe893fa7b13bb99d78c'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:02.565193 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-6c14f87f-9413-43fb-820e-b7a6fb7db210 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a95299c9dd894fe893fa7b13bb99d78c'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:02.583833 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-6c14f87f-9413-43fb-820e-b7a6fb7db210 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a95299c9dd894fe893fa7b13bb99d78c'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:02.597339 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-6c14f87f-9413-43fb-820e-b7a6fb7db210 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a95299c9dd894fe893fa7b13bb99d78c'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:02.598243 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-6c14f87f-9413-43fb-820e-b7a6fb7db210 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a95299c9dd894fe893fa7b13bb99d78c'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:02.611079 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1006/1995] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 9 12:14:02 2022] DELETE /identity/v3/users/a95299c9dd894fe893fa7b13bb99d78c => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:14:03.048424 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a62e80-c176-4c9e-bf30-0e6aff8b93c1 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:03.051192 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-f5a62e80-c176-4c9e-bf30-0e6aff8b93c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:03.071346 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a62e80-c176-4c9e-bf30-0e6aff8b93c1 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:03.134456 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8046c0e-7ad3-4791-a02a-b946be6aafca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:03.135108 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8046c0e-7ad3-4791-a02a-b946be6aafca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:03.135387 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8046c0e-7ad3-4791-a02a-b946be6aafca admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:03.135657 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8046c0e-7ad3-4791-a02a-b946be6aafca admin admin] PATH_INFO: `/v3/projects/f7215d2b1c864f429e926467b5bb0f95` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:03.137057 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8046c0e-7ad3-4791-a02a-b946be6aafca admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7215d2b1c864f429e926467b5bb0f95)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:03.139008 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8046c0e-7ad3-4791-a02a-b946be6aafca admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:03.152372 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:14:03.201387 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-b8046c0e-7ad3-4791-a02a-b946be6aafca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7215d2b1c864f429e926467b5bb0f95', 'request_id': 'req-b8046c0e-7ad3-4791-a02a-b946be6aafca'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:03.207047 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-b8046c0e-7ad3-4791-a02a-b946be6aafca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7215d2b1c864f429e926467b5bb0f95', 'request_id': 'req-b8046c0e-7ad3-4791-a02a-b946be6aafca'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:03.215466 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-b8046c0e-7ad3-4791-a02a-b946be6aafca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7215d2b1c864f429e926467b5bb0f95', 'request_id': 'req-b8046c0e-7ad3-4791-a02a-b946be6aafca'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:03.226840 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-b8046c0e-7ad3-4791-a02a-b946be6aafca admin admin] The token cache is being invalidate because project f7215d2b1c864f429e926467b5bb0f95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:14:03.227243 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-b8046c0e-7ad3-4791-a02a-b946be6aafca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:03.228063 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-b8046c0e-7ad3-4791-a02a-b946be6aafca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:03.229786 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 990/1996] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 9 12:14:03 2022] DELETE /identity/v3/projects/f7215d2b1c864f429e926467b5bb0f95 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:14:03.234392 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9732d1-8362-4a8a-8a86-72d2037e12a2 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:03.234679 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9732d1-8362-4a8a-8a86-72d2037e12a2 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:03.234979 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9732d1-8362-4a8a-8a86-72d2037e12a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:03.254617 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-fc9732d1-8362-4a8a-8a86-72d2037e12a2 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:14:03.256014 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-fc9732d1-8362-4a8a-8a86-72d2037e12a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:03.304386 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:14:03.323908 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1007/1997] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:14:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:14:03.332851 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df98f6c6-96c5-4d25-8041-9ecdfd74d6df None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:03.333133 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df98f6c6-96c5-4d25-8041-9ecdfd74d6df None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:03.333405 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df98f6c6-96c5-4d25-8041-9ecdfd74d6df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:03.354528 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-df98f6c6-96c5-4d25-8041-9ecdfd74d6df None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:14:03.355775 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-df98f6c6-96c5-4d25-8041-9ecdfd74d6df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:03.363825 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 991/1998] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:14:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:14:03.368105 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9732d1-8362-4a8a-8a86-72d2037e12a2 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:03.380427 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9732d1-8362-4a8a-8a86-72d2037e12a2 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:03.388416 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f898a029-135f-4218-9581-b7782aeeca1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:03.389181 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f898a029-135f-4218-9581-b7782aeeca1b admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:03.389464 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f898a029-135f-4218-9581-b7782aeeca1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:03.389732 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f898a029-135f-4218-9581-b7782aeeca1b admin admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:03.390719 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f898a029-135f-4218-9581-b7782aeeca1b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:14:03.390924 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f898a029-135f-4218-9581-b7782aeeca1b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:03.392819 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f898a029-135f-4218-9581-b7782aeeca1b admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:03.406277 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1008/1999] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 9 12:14:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:14:03.411045 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df98f6c6-96c5-4d25-8041-9ecdfd74d6df None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:03.425904 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df98f6c6-96c5-4d25-8041-9ecdfd74d6df None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:03.430533 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b91ff0-0bab-4ed1-a568-55885e295945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:03.431974 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b91ff0-0bab-4ed1-a568-55885e295945 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:03.432178 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b91ff0-0bab-4ed1-a568-55885e295945 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:03.432418 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b91ff0-0bab-4ed1-a568-55885e295945 admin admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:03.433047 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b91ff0-0bab-4ed1-a568-55885e295945 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:03.435360 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b91ff0-0bab-4ed1-a568-55885e295945 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:03.469059 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 992/2000] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 9 12:14:03 2022] POST /identity/v3/projects => generated 365 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:14:03.473303 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f898a029-135f-4218-9581-b7782aeeca1b admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:03.484960 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f898a029-135f-4218-9581-b7782aeeca1b admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:03.582218 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7a7153-9b6f-4336-a013-85fc84440bb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:03.582833 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7a7153-9b6f-4336-a013-85fc84440bb7 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:03.583278 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7a7153-9b6f-4336-a013-85fc84440bb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:03.583580 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7a7153-9b6f-4336-a013-85fc84440bb7 admin admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:03.584179 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7a7153-9b6f-4336-a013-85fc84440bb7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:03.586134 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7a7153-9b6f-4336-a013-85fc84440bb7 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:03.607890 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1009/2001] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:14:03 2022] POST /identity/v3/users => generated 283 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:14:03.611754 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b91ff0-0bab-4ed1-a568-55885e295945 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:03.622461 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b91ff0-0bab-4ed1-a568-55885e295945 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:03.627208 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644c0eb4-edec-4d23-a470-90cdb3a553cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:03.627949 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644c0eb4-edec-4d23-a470-90cdb3a553cc admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:03.628227 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644c0eb4-edec-4d23-a470-90cdb3a553cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:03.628497 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644c0eb4-edec-4d23-a470-90cdb3a553cc admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:03.629164 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644c0eb4-edec-4d23-a470-90cdb3a553cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:03.630868 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644c0eb4-edec-4d23-a470-90cdb3a553cc admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:03.642585 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 993/2002] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:14:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:14:03.646646 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7a7153-9b6f-4336-a013-85fc84440bb7 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:03.658419 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7a7153-9b6f-4336-a013-85fc84440bb7 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:03.663326 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b74383-2cba-4e77-8cce-e1d795c1ad0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:03.663967 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b74383-2cba-4e77-8cce-e1d795c1ad0e admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:03.664309 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b74383-2cba-4e77-8cce-e1d795c1ad0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:03.664583 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b74383-2cba-4e77-8cce-e1d795c1ad0e admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:03.665305 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b74383-2cba-4e77-8cce-e1d795c1ad0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:03.666969 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b74383-2cba-4e77-8cce-e1d795c1ad0e admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:03.677800 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1010/2003] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:14:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:14:03.682534 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644c0eb4-edec-4d23-a470-90cdb3a553cc admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:03.694075 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644c0eb4-edec-4d23-a470-90cdb3a553cc admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:03.699474 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c07a66-2cb7-46cf-ae55-bc393aa2dda4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:03.700158 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c07a66-2cb7-46cf-ae55-bc393aa2dda4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:03.700433 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c07a66-2cb7-46cf-ae55-bc393aa2dda4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:03.700707 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c07a66-2cb7-46cf-ae55-bc393aa2dda4 admin admin] PATH_INFO: `/v3/projects/5c082fded13e4c5399f3ce0d6778e7d8/users/1f5cd4165918469d978d269db2b46599/roles/eecd2f160dd844f5906f3d345780c501` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:03.716655 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c07a66-2cb7-46cf-ae55-bc393aa2dda4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c082fded13e4c5399f3ce0d6778e7d8, user_id=1f5cd4165918469d978d269db2b46599, role_id=eecd2f160dd844f5906f3d345780c501)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:03.720134 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c07a66-2cb7-46cf-ae55-bc393aa2dda4 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:03.733634 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 994/2004] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:14:03 2022] PUT /identity/v3/projects/5c082fded13e4c5399f3ce0d6778e7d8/users/1f5cd4165918469d978d269db2b46599/roles/eecd2f160dd844f5906f3d345780c501 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:14:03.745661 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7624ebbe-4fc8-4704-a31b-91c9e5028846 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:03.746072 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7624ebbe-4fc8-4704-a31b-91c9e5028846 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:03.746358 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7624ebbe-4fc8-4704-a31b-91c9e5028846 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:03.775989 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-7624ebbe-4fc8-4704-a31b-91c9e5028846 None None] MFA Rules not processed for user `1f5cd4165918469d978d269db2b46599`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:14:03.777207 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-7624ebbe-4fc8-4704-a31b-91c9e5028846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:03.841396 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1011/2005] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:14:03 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:14:03.845880 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c07a66-2cb7-46cf-ae55-bc393aa2dda4 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:03.907356 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c07a66-2cb7-46cf-ae55-bc393aa2dda4 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:03.912188 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3d6295-0f87-4249-9577-e764a3257fd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:03.912785 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3d6295-0f87-4249-9577-e764a3257fd5 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:03.913065 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3d6295-0f87-4249-9577-e764a3257fd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:03.913332 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3d6295-0f87-4249-9577-e764a3257fd5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:03.913950 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3d6295-0f87-4249-9577-e764a3257fd5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:03.916762 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3d6295-0f87-4249-9577-e764a3257fd5 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:03.944855 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 995/2006] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 9 12:14:03 2022] POST /identity/v3/projects => generated 365 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:14:03.949548 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7624ebbe-4fc8-4704-a31b-91c9e5028846 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:03.960144 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7624ebbe-4fc8-4704-a31b-91c9e5028846 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:04.009578 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:14:04.026805 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3d6295-0f87-4249-9577-e764a3257fd5 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:04.029184 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3afc211-0cd0-4793-816c-19c570d7f515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:04.030074 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3afc211-0cd0-4793-816c-19c570d7f515 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:04.030714 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3afc211-0cd0-4793-816c-19c570d7f515 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:04.031244 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3afc211-0cd0-4793-816c-19c570d7f515 admin admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:04.031470 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-7d3d6295-0f87-4249-9577-e764a3257fd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:04.032413 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3afc211-0cd0-4793-816c-19c570d7f515 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:04.034655 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3afc211-0cd0-4793-816c-19c570d7f515 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:04.047313 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3d6295-0f87-4249-9577-e764a3257fd5 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:04.053094 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68920362-bd91-4ead-b0f0-625fe93076cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:04.053735 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68920362-bd91-4ead-b0f0-625fe93076cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:04.054111 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68920362-bd91-4ead-b0f0-625fe93076cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:04.054299 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68920362-bd91-4ead-b0f0-625fe93076cf admin admin] PATH_INFO: `/v3/users/71720a8873914d198d72a0ddd2c17deb` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:04.058930 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68920362-bd91-4ead-b0f0-625fe93076cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=71720a8873914d198d72a0ddd2c17deb)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:04.060776 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68920362-bd91-4ead-b0f0-625fe93076cf admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:04.065944 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1012/2007] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:14:03 2022] POST /identity/v3/users => generated 289 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:14:04.069696 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3afc211-0cd0-4793-816c-19c570d7f515 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:04.078427 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=88,keystone:DELETE=8,keystone:INSERT=4 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:14:04.093460 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3afc211-0cd0-4793-816c-19c570d7f515 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:04.100794 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1575e7f2-66f5-4853-bf80-f633fdc24e18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:04.101447 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1575e7f2-66f5-4853-bf80-f633fdc24e18 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:04.104193 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1575e7f2-66f5-4853-bf80-f633fdc24e18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:04.104528 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1575e7f2-66f5-4853-bf80-f633fdc24e18 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:04.105247 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1575e7f2-66f5-4853-bf80-f633fdc24e18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:04.107774 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1575e7f2-66f5-4853-bf80-f633fdc24e18 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:04.122885 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1013/2008] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:14:04 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:14:04.123965 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1575e7f2-66f5-4853-bf80-f633fdc24e18 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:04.135259 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1575e7f2-66f5-4853-bf80-f633fdc24e18 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:04.139910 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-68920362-bd91-4ead-b0f0-625fe93076cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71720a8873914d198d72a0ddd2c17deb'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:04.140160 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c02de26-94af-4ec7-a384-ada3a0989066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:04.140828 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c02de26-94af-4ec7-a384-ada3a0989066 admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:04.141116 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c02de26-94af-4ec7-a384-ada3a0989066 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:04.141394 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c02de26-94af-4ec7-a384-ada3a0989066 admin admin] PATH_INFO: `/v3/projects/f3d1e4d790514a198e8d838633700558/users/29758757a0b24cd4abf69f87ea9751f2/roles/3f2ca33902a04b6dbc2ef7d06d7289d3` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:04.162585 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c02de26-94af-4ec7-a384-ada3a0989066 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3d1e4d790514a198e8d838633700558, user_id=29758757a0b24cd4abf69f87ea9751f2, role_id=3f2ca33902a04b6dbc2ef7d06d7289d3)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:04.164533 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-68920362-bd91-4ead-b0f0-625fe93076cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71720a8873914d198d72a0ddd2c17deb'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:04.166840 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c02de26-94af-4ec7-a384-ada3a0989066 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:04.173750 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-68920362-bd91-4ead-b0f0-625fe93076cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71720a8873914d198d72a0ddd2c17deb'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:04.183291 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-68920362-bd91-4ead-b0f0-625fe93076cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71720a8873914d198d72a0ddd2c17deb'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:04.183991 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-68920362-bd91-4ead-b0f0-625fe93076cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71720a8873914d198d72a0ddd2c17deb'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:04.185171 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1014/2009] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:14:04 2022] PUT /identity/v3/projects/f3d1e4d790514a198e8d838633700558/users/29758757a0b24cd4abf69f87ea9751f2/roles/3f2ca33902a04b6dbc2ef7d06d7289d3 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:14:04.188750 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c02de26-94af-4ec7-a384-ada3a0989066 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:04.190391 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-2c02de26-94af-4ec7-a384-ada3a0989066 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:04.193314 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 996/2010] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 9 12:14:04 2022] DELETE /identity/v3/users/71720a8873914d198d72a0ddd2c17deb => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:14:04.194865 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728a43a6-c457-4a32-9037-8d382166dfc8 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:04.195396 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728a43a6-c457-4a32-9037-8d382166dfc8 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:04.195678 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728a43a6-c457-4a32-9037-8d382166dfc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:04.218451 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-728a43a6-c457-4a32-9037-8d382166dfc8 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:14:04.219775 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-728a43a6-c457-4a32-9037-8d382166dfc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:04.265113 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c02de26-94af-4ec7-a384-ada3a0989066 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:04.288711 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 997/2011] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:14:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:14:04.299017 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728a43a6-c457-4a32-9037-8d382166dfc8 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:04.301194 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-728a43a6-c457-4a32-9037-8d382166dfc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:04.334565 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9890f1-7d1e-41e9-a555-4609a8fe8cd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:04.335213 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9890f1-7d1e-41e9-a555-4609a8fe8cd9 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:04.335493 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9890f1-7d1e-41e9-a555-4609a8fe8cd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:04.335908 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9890f1-7d1e-41e9-a555-4609a8fe8cd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:04.336558 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9890f1-7d1e-41e9-a555-4609a8fe8cd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:04.338870 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9890f1-7d1e-41e9-a555-4609a8fe8cd9 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:04.353898 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1015/2012] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:14:04 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 12:14:04.355226 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9890f1-7d1e-41e9-a555-4609a8fe8cd9 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:04.368172 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9890f1-7d1e-41e9-a555-4609a8fe8cd9 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:04.369683 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728a43a6-c457-4a32-9037-8d382166dfc8 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:04.372918 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a897e7-d2bb-43aa-9933-e42d06503a92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:04.373587 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a897e7-d2bb-43aa-9933-e42d06503a92 admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:04.373765 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02899b5-a58a-43c2-af96-28a1c102c7f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:04.373885 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a897e7-d2bb-43aa-9933-e42d06503a92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:04.374122 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a897e7-d2bb-43aa-9933-e42d06503a92 admin admin] PATH_INFO: `/v3/projects/f3d1e4d790514a198e8d838633700558/users/29758757a0b24cd4abf69f87ea9751f2/roles/3f2ca33902a04b6dbc2ef7d06d7289d3` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:04.374358 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02899b5-a58a-43c2-af96-28a1c102c7f8 service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:04.374641 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02899b5-a58a-43c2-af96-28a1c102c7f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:04.374925 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02899b5-a58a-43c2-af96-28a1c102c7f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:04.376423 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a897e7-d2bb-43aa-9933-e42d06503a92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3d1e4d790514a198e8d838633700558, user_id=29758757a0b24cd4abf69f87ea9751f2, role_id=3f2ca33902a04b6dbc2ef7d06d7289d3)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:04.379104 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a897e7-d2bb-43aa-9933-e42d06503a92 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:04.385806 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02899b5-a58a-43c2-af96-28a1c102c7f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:04.387293 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02899b5-a58a-43c2-af96-28a1c102c7f8 service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:04.390632 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1016/2013] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:14:04 2022] PUT /identity/v3/projects/f3d1e4d790514a198e8d838633700558/users/29758757a0b24cd4abf69f87ea9751f2/roles/3f2ca33902a04b6dbc2ef7d06d7289d3 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:14:04.397373 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 998/2014] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:14:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:14:04.401574 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338f91e7-7a48-4a8c-8967-b0ff6ff673f6 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:04.401857 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338f91e7-7a48-4a8c-8967-b0ff6ff673f6 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:04.402130 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338f91e7-7a48-4a8c-8967-b0ff6ff673f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:04.435569 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-338f91e7-7a48-4a8c-8967-b0ff6ff673f6 None None] MFA Rules not processed for user `29758757a0b24cd4abf69f87ea9751f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:14:04.436950 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-338f91e7-7a48-4a8c-8967-b0ff6ff673f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:04.463061 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:14:04.524904 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1017/2015] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:14:04 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:14:04.533141 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02899b5-a58a-43c2-af96-28a1c102c7f8 service neutron] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:04.534632 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-d02899b5-a58a-43c2-af96-28a1c102c7f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:04.596582 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:14:04.645173 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02899b5-a58a-43c2-af96-28a1c102c7f8 service neutron] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:04.722446 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a874526e-1b5b-4d04-9b24-37633e2737e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535c258100df40b58a5ee44b2c21d621', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:04.723875 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a874526e-1b5b-4d04-9b24-37633e2737e8 service cinder] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:04.724156 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a874526e-1b5b-4d04-9b24-37633e2737e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:04.724627 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a874526e-1b5b-4d04-9b24-37633e2737e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:04.735402 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338f91e7-7a48-4a8c-8967-b0ff6ff673f6 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:04.737571 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-338f91e7-7a48-4a8c-8967-b0ff6ff673f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:04.746254 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a874526e-1b5b-4d04-9b24-37633e2737e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:04.747822 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a874526e-1b5b-4d04-9b24-37633e2737e8 service cinder] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:04.787091 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 999/2016] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:14:04 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:14:04.846041 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338f91e7-7a48-4a8c-8967-b0ff6ff673f6 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:04.857319 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f7b18b-70ca-4b8c-9765-241a9b8853c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:04.858192 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f7b18b-70ca-4b8c-9765-241a9b8853c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:04.858482 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f7b18b-70ca-4b8c-9765-241a9b8853c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:04.858799 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f7b18b-70ca-4b8c-9765-241a9b8853c7 admin admin] PATH_INFO: `/v3/projects/3b1160d8bfc942caab5d6841cbc695a1` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:04.860397 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f7b18b-70ca-4b8c-9765-241a9b8853c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b1160d8bfc942caab5d6841cbc695a1)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:04.862495 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f7b18b-70ca-4b8c-9765-241a9b8853c7 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:04.884600 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a874526e-1b5b-4d04-9b24-37633e2737e8 service cinder] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:04.894283 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a874526e-1b5b-4d04-9b24-37633e2737e8 service cinder] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:04.898154 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dd9dcf-33b3-4939-9f15-db339d919242 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535c258100df40b58a5ee44b2c21d621', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:04.898760 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dd9dcf-33b3-4939-9f15-db339d919242 service cinder] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:04.899075 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dd9dcf-33b3-4939-9f15-db339d919242 service cinder] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:04.899425 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dd9dcf-33b3-4939-9f15-db339d919242 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:04.901729 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-21dd9dcf-33b3-4939-9f15-db339d919242 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:04.916874 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-76f7b18b-70ca-4b8c-9765-241a9b8853c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b1160d8bfc942caab5d6841cbc695a1', 'request_id': 'req-76f7b18b-70ca-4b8c-9765-241a9b8853c7'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:04.923555 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-76f7b18b-70ca-4b8c-9765-241a9b8853c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b1160d8bfc942caab5d6841cbc695a1', 'request_id': 'req-76f7b18b-70ca-4b8c-9765-241a9b8853c7'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:04.931905 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-76f7b18b-70ca-4b8c-9765-241a9b8853c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b1160d8bfc942caab5d6841cbc695a1', 'request_id': 'req-76f7b18b-70ca-4b8c-9765-241a9b8853c7'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:04.951308 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-76f7b18b-70ca-4b8c-9765-241a9b8853c7 admin admin] The token cache is being invalidate because project 3b1160d8bfc942caab5d6841cbc695a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81591) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:14:04.951901 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-76f7b18b-70ca-4b8c-9765-241a9b8853c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:04.952760 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-76f7b18b-70ca-4b8c-9765-241a9b8853c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:04.955234 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1018/2017] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 9 12:14:04 2022] DELETE /identity/v3/projects/3b1160d8bfc942caab5d6841cbc695a1 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:14:04.960322 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4c5fa4-0053-4c99-a6c8-08f7261fab9e None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:04.960796 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4c5fa4-0053-4c99-a6c8-08f7261fab9e None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:04.961241 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4c5fa4-0053-4c99-a6c8-08f7261fab9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:04.972442 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dd9dcf-33b3-4939-9f15-db339d919242 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:04.973969 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dd9dcf-33b3-4939-9f15-db339d919242 service cinder] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:04.982721 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-2b4c5fa4-0053-4c99-a6c8-08f7261fab9e None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:14:04.983980 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-2b4c5fa4-0053-4c99-a6c8-08f7261fab9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:04.984563 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1000/2018] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:14:04 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:14:05.030512 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:14:05.057676 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1019/2019] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:14:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:14:05.066202 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbcf3b7-5f89-4151-9d85-730ffd6d30b0 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:05.066727 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbcf3b7-5f89-4151-9d85-730ffd6d30b0 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:05.066812 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbcf3b7-5f89-4151-9d85-730ffd6d30b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:05.090316 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-8fbcf3b7-5f89-4151-9d85-730ffd6d30b0 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:14:05.091629 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-8fbcf3b7-5f89-4151-9d85-730ffd6d30b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:05.100135 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1001/2020] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:14:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:14:05.104567 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4c5fa4-0053-4c99-a6c8-08f7261fab9e None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:05.116621 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4c5fa4-0053-4c99-a6c8-08f7261fab9e None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:05.123015 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b532c6-b272-40f8-9be9-7d5318809cf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:05.123685 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b532c6-b272-40f8-9be9-7d5318809cf1 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:05.123966 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b532c6-b272-40f8-9be9-7d5318809cf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:05.124237 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b532c6-b272-40f8-9be9-7d5318809cf1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:05.124890 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b532c6-b272-40f8-9be9-7d5318809cf1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:14:05.125245 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b532c6-b272-40f8-9be9-7d5318809cf1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:05.126863 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b532c6-b272-40f8-9be9-7d5318809cf1 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:05.154579 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1020/2021] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 9 12:14:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:14:05.159928 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbcf3b7-5f89-4151-9d85-730ffd6d30b0 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:05.172484 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbcf3b7-5f89-4151-9d85-730ffd6d30b0 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:05.178279 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e96c04-a173-4197-bc49-450fb9b025f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:05.179112 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e96c04-a173-4197-bc49-450fb9b025f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:05.179461 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e96c04-a173-4197-bc49-450fb9b025f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:05.179746 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e96c04-a173-4197-bc49-450fb9b025f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:05.182032 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e96c04-a173-4197-bc49-450fb9b025f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:05.183416 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e96c04-a173-4197-bc49-450fb9b025f3 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:05.215174 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1002/2022] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 9 12:14:05 2022] POST /identity/v3/projects => generated 383 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:14:05.218493 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b532c6-b272-40f8-9be9-7d5318809cf1 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:05.229499 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b532c6-b272-40f8-9be9-7d5318809cf1 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:05.305101 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76db4515-b967-49a8-9efc-427a41aeb8f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:05.306403 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76db4515-b967-49a8-9efc-427a41aeb8f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:05.306749 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76db4515-b967-49a8-9efc-427a41aeb8f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:05.307132 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76db4515-b967-49a8-9efc-427a41aeb8f8 admin admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:05.308277 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76db4515-b967-49a8-9efc-427a41aeb8f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:05.311397 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76db4515-b967-49a8-9efc-427a41aeb8f8 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:05.342565 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1021/2023] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:14:05 2022] POST /identity/v3/users => generated 292 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:14:05.346956 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e96c04-a173-4197-bc49-450fb9b025f3 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:05.358244 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e96c04-a173-4197-bc49-450fb9b025f3 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:05.363280 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee65926-4469-4ae5-b611-6e405d2b3dde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:05.363911 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee65926-4469-4ae5-b611-6e405d2b3dde admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:05.364190 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee65926-4469-4ae5-b611-6e405d2b3dde admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:05.364459 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee65926-4469-4ae5-b611-6e405d2b3dde admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:05.365191 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee65926-4469-4ae5-b611-6e405d2b3dde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:05.366811 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee65926-4469-4ae5-b611-6e405d2b3dde admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:05.378598 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1003/2024] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:14:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:14:05.382521 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76db4515-b967-49a8-9efc-427a41aeb8f8 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:05.393641 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76db4515-b967-49a8-9efc-427a41aeb8f8 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:05.398343 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d085c7a7-ee91-4a91-9d8f-4c9fce68fbf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:05.398981 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d085c7a7-ee91-4a91-9d8f-4c9fce68fbf7 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:05.399260 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d085c7a7-ee91-4a91-9d8f-4c9fce68fbf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:05.399556 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d085c7a7-ee91-4a91-9d8f-4c9fce68fbf7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:05.400208 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d085c7a7-ee91-4a91-9d8f-4c9fce68fbf7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:05.401802 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d085c7a7-ee91-4a91-9d8f-4c9fce68fbf7 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:05.413574 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1022/2025] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:14:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:14:05.417726 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee65926-4469-4ae5-b611-6e405d2b3dde admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:05.428676 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee65926-4469-4ae5-b611-6e405d2b3dde admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:05.434094 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6075c2-9885-42ec-8be2-c1188dec339e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:05.434767 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6075c2-9885-42ec-8be2-c1188dec339e admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:05.435355 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6075c2-9885-42ec-8be2-c1188dec339e admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:05.435631 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6075c2-9885-42ec-8be2-c1188dec339e admin admin] PATH_INFO: `/v3/projects/eef8dffce9a145b58a532bf9eac3c36c/users/648b8ea6500746e6b68def6fe5883eb4/roles/eecd2f160dd844f5906f3d345780c501` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:05.452011 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6075c2-9885-42ec-8be2-c1188dec339e admin admin] RBAC: Authorizing `identity:create_grant(project_id=eef8dffce9a145b58a532bf9eac3c36c, user_id=648b8ea6500746e6b68def6fe5883eb4, role_id=eecd2f160dd844f5906f3d345780c501)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:05.454681 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6075c2-9885-42ec-8be2-c1188dec339e admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:05.468779 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1004/2026] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:14:05 2022] PUT /identity/v3/projects/eef8dffce9a145b58a532bf9eac3c36c/users/648b8ea6500746e6b68def6fe5883eb4/roles/eecd2f160dd844f5906f3d345780c501 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:14:05.479845 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a98e4c3-873d-4163-87c5-7d9a0d7f8ace None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:05.480361 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a98e4c3-873d-4163-87c5-7d9a0d7f8ace None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:05.480361 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a98e4c3-873d-4163-87c5-7d9a0d7f8ace None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:05.508152 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-2a98e4c3-873d-4163-87c5-7d9a0d7f8ace None None] MFA Rules not processed for user `648b8ea6500746e6b68def6fe5883eb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:14:05.509637 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-2a98e4c3-873d-4163-87c5-7d9a0d7f8ace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:05.575345 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1023/2027] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:14:05 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:14:05.588617 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6075c2-9885-42ec-8be2-c1188dec339e admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:05.590374 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-5c6075c2-9885-42ec-8be2-c1188dec339e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:05.626515 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:14:05.653607 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6075c2-9885-42ec-8be2-c1188dec339e admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:05.657351 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c1efaf-8ff9-4465-ac42-9f3123bb7a47 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535c258100df40b58a5ee44b2c21d621', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:05.657964 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c1efaf-8ff9-4465-ac42-9f3123bb7a47 service cinder] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:05.658242 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c1efaf-8ff9-4465-ac42-9f3123bb7a47 service cinder] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:05.658514 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c1efaf-8ff9-4465-ac42-9f3123bb7a47 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:05.667774 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c1efaf-8ff9-4465-ac42-9f3123bb7a47 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:05.669219 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c1efaf-8ff9-4465-ac42-9f3123bb7a47 service cinder] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:05.678367 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1005/2028] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:14:05 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:14:08.315497 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a98e4c3-873d-4163-87c5-7d9a0d7f8ace None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:08.391323 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:14:08.425329 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a98e4c3-873d-4163-87c5-7d9a0d7f8ace None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:08.432531 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6788cb4e-4e5d-489a-af2f-74c57b52febe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:08.433392 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6788cb4e-4e5d-489a-af2f-74c57b52febe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:08.433921 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6788cb4e-4e5d-489a-af2f-74c57b52febe admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:08.434274 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6788cb4e-4e5d-489a-af2f-74c57b52febe admin admin] PATH_INFO: `/v3/users/648b8ea6500746e6b68def6fe5883eb4` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:08.435992 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6788cb4e-4e5d-489a-af2f-74c57b52febe admin admin] RBAC: Authorizing `identity:delete_user(user_id=648b8ea6500746e6b68def6fe5883eb4)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:08.438109 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6788cb4e-4e5d-489a-af2f-74c57b52febe admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:08.545550 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-6788cb4e-4e5d-489a-af2f-74c57b52febe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '648b8ea6500746e6b68def6fe5883eb4'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:08.563785 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-6788cb4e-4e5d-489a-af2f-74c57b52febe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '648b8ea6500746e6b68def6fe5883eb4'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:08.579928 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-6788cb4e-4e5d-489a-af2f-74c57b52febe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '648b8ea6500746e6b68def6fe5883eb4'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:08.593122 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-6788cb4e-4e5d-489a-af2f-74c57b52febe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '648b8ea6500746e6b68def6fe5883eb4'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:08.594191 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-6788cb4e-4e5d-489a-af2f-74c57b52febe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '648b8ea6500746e6b68def6fe5883eb4'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:08.619720 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1024/2029] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 9 12:14:08 2022] DELETE /identity/v3/users/648b8ea6500746e6b68def6fe5883eb4 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:14:08.632575 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6141a5-bebb-4b0f-b886-878ffa162b18 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:08.632780 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c244488e-57fa-4ccc-93d7-fcbc62a5004d None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:08.633242 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6141a5-bebb-4b0f-b886-878ffa162b18 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:08.634291 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6141a5-bebb-4b0f-b886-878ffa162b18 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:08.636063 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1025/2030] 10.222.0.49 () {54 vars in 921 bytes} [Fri Sep 9 12:14:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:14:08.636431 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c244488e-57fa-4ccc-93d7-fcbc62a5004d None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:08.636717 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c244488e-57fa-4ccc-93d7-fcbc62a5004d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:08.658955 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2475e185-643e-43cf-a363-844335494563 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:08.659389 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2475e185-643e-43cf-a363-844335494563 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:08.660330 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2475e185-643e-43cf-a363-844335494563 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:08.666563 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-c244488e-57fa-4ccc-93d7-fcbc62a5004d None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:14:08.667814 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-c244488e-57fa-4ccc-93d7-fcbc62a5004d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:08.684103 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-2475e185-643e-43cf-a363-844335494563 None None] MFA Rules not processed for user `d3adb2f0583c4ff782049dacdb3ee556`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:14:08.685789 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-2475e185-643e-43cf-a363-844335494563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:08.767683 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1026/2031] 10.222.0.49 () {58 vars in 1018 bytes} [Fri Sep 9 12:14:08 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:14:08.768488 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2475e185-643e-43cf-a363-844335494563 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:08.769839 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-2475e185-643e-43cf-a363-844335494563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:08.775535 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1006/2032] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:14:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:14:08.783426 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8be96b-9101-4a3a-80e2-c8b288ffa2fa None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:08.783821 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8be96b-9101-4a3a-80e2-c8b288ffa2fa None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:08.784799 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8be96b-9101-4a3a-80e2-c8b288ffa2fa None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:08.785799 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1007/2033] 10.222.0.49 () {54 vars in 961 bytes} [Fri Sep 9 12:14:08 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:14:08.787572 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8be96b-9101-4a3a-80e2-c8b288ffa2fa None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:08.789202 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-5c8be96b-9101-4a3a-80e2-c8b288ffa2fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:08.799755 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2475e185-643e-43cf-a363-844335494563 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:08.806127 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b8375b-f65b-4351-8405-6c774aba1f17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:08.806765 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b8375b-f65b-4351-8405-6c774aba1f17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:08.807073 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b8375b-f65b-4351-8405-6c774aba1f17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:08.807339 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b8375b-f65b-4351-8405-6c774aba1f17 admin admin] PATH_INFO: `/v3/users/ab948442f326433c8e9f5f90314627aa` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:08.808676 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b8375b-f65b-4351-8405-6c774aba1f17 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab948442f326433c8e9f5f90314627aa)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:08.810983 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b8375b-f65b-4351-8405-6c774aba1f17 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:08.856279 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8be96b-9101-4a3a-80e2-c8b288ffa2fa None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:08.860008 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-d7b8375b-f65b-4351-8405-6c774aba1f17 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab948442f326433c8e9f5f90314627aa'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:08.860157 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd83f6ab-9647-434b-a918-357252cfe76c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:08.860770 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd83f6ab-9647-434b-a918-357252cfe76c service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:08.861049 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd83f6ab-9647-434b-a918-357252cfe76c service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:08.861347 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd83f6ab-9647-434b-a918-357252cfe76c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:08.864522 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:14:08.867732 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-d7b8375b-f65b-4351-8405-6c774aba1f17 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab948442f326433c8e9f5f90314627aa'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:08.870815 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd83f6ab-9647-434b-a918-357252cfe76c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:08.872424 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd83f6ab-9647-434b-a918-357252cfe76c service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:08.881985 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1008/2034] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:14:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:14:08.883204 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caa0bb6-a578-4e8e-a590-2c6e7e1cc676 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:08.883536 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caa0bb6-a578-4e8e-a590-2c6e7e1cc676 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:08.883801 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caa0bb6-a578-4e8e-a590-2c6e7e1cc676 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:08.883922 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-d7b8375b-f65b-4351-8405-6c774aba1f17 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab948442f326433c8e9f5f90314627aa'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:08.893568 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-d7b8375b-f65b-4351-8405-6c774aba1f17 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab948442f326433c8e9f5f90314627aa'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:08.894239 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-d7b8375b-f65b-4351-8405-6c774aba1f17 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab948442f326433c8e9f5f90314627aa'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:08.902367 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1027/2035] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 9 12:14:08 2022] DELETE /identity/v3/users/ab948442f326433c8e9f5f90314627aa => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:14:08.903976 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-0caa0bb6-a578-4e8e-a590-2c6e7e1cc676 None None] MFA Rules not processed for user `d3adb2f0583c4ff782049dacdb3ee556`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:14:08.905180 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-0caa0bb6-a578-4e8e-a590-2c6e7e1cc676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:09.003477 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1009/2036] 10.222.0.49 () {58 vars in 1058 bytes} [Fri Sep 9 12:14:08 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:14:09.012014 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b8375b-f65b-4351-8405-6c774aba1f17 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:09.025107 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b8375b-f65b-4351-8405-6c774aba1f17 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:09.031964 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7209eaad-0bd5-4e8b-be51-3ef704e8a46b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3adb2f0583c4ff782049dacdb3ee556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:09.032638 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7209eaad-0bd5-4e8b-be51-3ef704e8a46b service placement] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:09.032819 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7209eaad-0bd5-4e8b-be51-3ef704e8a46b service placement] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:09.033043 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7209eaad-0bd5-4e8b-be51-3ef704e8a46b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:09.034998 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-7209eaad-0bd5-4e8b-be51-3ef704e8a46b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:09.048285 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7209eaad-0bd5-4e8b-be51-3ef704e8a46b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:09.049836 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7209eaad-0bd5-4e8b-be51-3ef704e8a46b service placement] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:09.065625 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1028/2037] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:14:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:14:09.188142 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caa0bb6-a578-4e8e-a590-2c6e7e1cc676 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:09.190010 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-0caa0bb6-a578-4e8e-a590-2c6e7e1cc676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:09.206125 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7209eaad-0bd5-4e8b-be51-3ef704e8a46b service placement] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:09.207932 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-7209eaad-0bd5-4e8b-be51-3ef704e8a46b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:09.232094 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:14:09.280774 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7209eaad-0bd5-4e8b-be51-3ef704e8a46b service placement] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:09.285504 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffa5882-54f0-4e7d-845d-7c8feff7b7a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:09.286059 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffa5882-54f0-4e7d-845d-7c8feff7b7a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:09.286334 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffa5882-54f0-4e7d-845d-7c8feff7b7a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:09.286604 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffa5882-54f0-4e7d-845d-7c8feff7b7a4 admin admin] PATH_INFO: `/v3/projects/eef8dffce9a145b58a532bf9eac3c36c` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:09.287742 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffa5882-54f0-4e7d-845d-7c8feff7b7a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eef8dffce9a145b58a532bf9eac3c36c)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:09.289664 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffa5882-54f0-4e7d-845d-7c8feff7b7a4 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:09.290080 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caa0bb6-a578-4e8e-a590-2c6e7e1cc676 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:09.294620 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ab07ba-2bbd-4ee0-a444-7415470cdd92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:09.295350 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ab07ba-2bbd-4ee0-a444-7415470cdd92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:09.295638 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ab07ba-2bbd-4ee0-a444-7415470cdd92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:09.295898 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ab07ba-2bbd-4ee0-a444-7415470cdd92 admin admin] PATH_INFO: `/v3/projects/433140f4c010464fa04a5622b26bc3ab` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:09.297011 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ab07ba-2bbd-4ee0-a444-7415470cdd92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=433140f4c010464fa04a5622b26bc3ab)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:09.298926 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ab07ba-2bbd-4ee0-a444-7415470cdd92 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:09.337966 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-2ffa5882-54f0-4e7d-845d-7c8feff7b7a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eef8dffce9a145b58a532bf9eac3c36c', 'request_id': 'req-2ffa5882-54f0-4e7d-845d-7c8feff7b7a4'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:09.344053 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-2ffa5882-54f0-4e7d-845d-7c8feff7b7a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eef8dffce9a145b58a532bf9eac3c36c', 'request_id': 'req-2ffa5882-54f0-4e7d-845d-7c8feff7b7a4'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:09.347172 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-d1ab07ba-2bbd-4ee0-a444-7415470cdd92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '433140f4c010464fa04a5622b26bc3ab', 'request_id': 'req-d1ab07ba-2bbd-4ee0-a444-7415470cdd92'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:09.350744 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-2ffa5882-54f0-4e7d-845d-7c8feff7b7a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eef8dffce9a145b58a532bf9eac3c36c', 'request_id': 'req-2ffa5882-54f0-4e7d-845d-7c8feff7b7a4'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:09.352875 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-d1ab07ba-2bbd-4ee0-a444-7415470cdd92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '433140f4c010464fa04a5622b26bc3ab', 'request_id': 'req-d1ab07ba-2bbd-4ee0-a444-7415470cdd92'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:09.362117 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-d1ab07ba-2bbd-4ee0-a444-7415470cdd92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '433140f4c010464fa04a5622b26bc3ab', 'request_id': 'req-d1ab07ba-2bbd-4ee0-a444-7415470cdd92'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:09.364570 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-2ffa5882-54f0-4e7d-845d-7c8feff7b7a4 admin admin] The token cache is being invalidate because project eef8dffce9a145b58a532bf9eac3c36c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81591) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:14:09.364854 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-2ffa5882-54f0-4e7d-845d-7c8feff7b7a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:09.365471 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-2ffa5882-54f0-4e7d-845d-7c8feff7b7a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:09.366977 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1029/2038] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 9 12:14:09 2022] DELETE /identity/v3/projects/eef8dffce9a145b58a532bf9eac3c36c => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:14:09.372258 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c2df6c-0535-41ca-8823-3873909492d0 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:09.372540 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c2df6c-0535-41ca-8823-3873909492d0 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:09.372836 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c2df6c-0535-41ca-8823-3873909492d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:09.374334 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-d1ab07ba-2bbd-4ee0-a444-7415470cdd92 admin admin] The token cache is being invalidate because project 433140f4c010464fa04a5622b26bc3ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:14:09.374683 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-d1ab07ba-2bbd-4ee0-a444-7415470cdd92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:09.375375 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-d1ab07ba-2bbd-4ee0-a444-7415470cdd92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:09.376861 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1010/2039] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 9 12:14:09 2022] DELETE /identity/v3/projects/433140f4c010464fa04a5622b26bc3ab => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:14:09.381304 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99007a7b-70b0-487f-bb16-64953ed36d84 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:09.381582 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99007a7b-70b0-487f-bb16-64953ed36d84 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:09.381852 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99007a7b-70b0-487f-bb16-64953ed36d84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:09.392901 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-52c2df6c-0535-41ca-8823-3873909492d0 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:14:09.394113 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-52c2df6c-0535-41ca-8823-3873909492d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:09.402059 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-99007a7b-70b0-487f-bb16-64953ed36d84 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:14:09.403309 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-99007a7b-70b0-487f-bb16-64953ed36d84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:09.437943 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:14:09.457115 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1030/2040] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:14:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:14:09.466760 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e087a6-2e5a-410c-86f3-45c402069c60 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:09.467086 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e087a6-2e5a-410c-86f3-45c402069c60 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:09.467386 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e087a6-2e5a-410c-86f3-45c402069c60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:09.467620 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1011/2041] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:14:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:14:09.474512 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ceb52f-595c-4c19-82c5-6aaf512612b7 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:09.474800 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ceb52f-595c-4c19-82c5-6aaf512612b7 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:09.475106 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ceb52f-595c-4c19-82c5-6aaf512612b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:09.487494 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-a7e087a6-2e5a-410c-86f3-45c402069c60 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:14:09.488702 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-a7e087a6-2e5a-410c-86f3-45c402069c60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:09.495922 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-e1ceb52f-595c-4c19-82c5-6aaf512612b7 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:14:09.497091 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-e1ceb52f-595c-4c19-82c5-6aaf512612b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:09.501601 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1031/2042] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:14:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:14:09.508373 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1012/2043] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:14:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:14:09.508534 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e087a6-2e5a-410c-86f3-45c402069c60 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:09.509353 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ceb52f-595c-4c19-82c5-6aaf512612b7 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:09.515135 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e087a6-2e5a-410c-86f3-45c402069c60 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:09.518755 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ceb52f-595c-4c19-82c5-6aaf512612b7 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:09.519806 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9a2dbe-a369-46b3-b166-71ea272960a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:09.520398 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9a2dbe-a369-46b3-b166-71ea272960a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:09.520671 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9a2dbe-a369-46b3-b166-71ea272960a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:09.520937 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9a2dbe-a369-46b3-b166-71ea272960a6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:09.521595 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9a2dbe-a369-46b3-b166-71ea272960a6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:14:09.521920 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9a2dbe-a369-46b3-b166-71ea272960a6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:09.523558 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9a2dbe-a369-46b3-b166-71ea272960a6 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:09.524100 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893d51ec-536e-4898-b9f6-27ef5f5d7c21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:09.524698 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893d51ec-536e-4898-b9f6-27ef5f5d7c21 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:09.525223 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893d51ec-536e-4898-b9f6-27ef5f5d7c21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:09.525501 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893d51ec-536e-4898-b9f6-27ef5f5d7c21 admin admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:09.526239 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893d51ec-536e-4898-b9f6-27ef5f5d7c21 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:14:09.526541 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893d51ec-536e-4898-b9f6-27ef5f5d7c21 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:09.528166 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893d51ec-536e-4898-b9f6-27ef5f5d7c21 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:09.535324 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1032/2044] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 9 12:14:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:14:09.539589 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9a2dbe-a369-46b3-b166-71ea272960a6 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:09.543458 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1013/2045] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 9 12:14:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:14:09.544329 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893d51ec-536e-4898-b9f6-27ef5f5d7c21 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:09.550575 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9a2dbe-a369-46b3-b166-71ea272960a6 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:09.554171 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893d51ec-536e-4898-b9f6-27ef5f5d7c21 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:09.555308 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9163a1f-4478-4927-97ab-f17933671141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:09.556571 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9163a1f-4478-4927-97ab-f17933671141 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:09.556769 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9163a1f-4478-4927-97ab-f17933671141 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:09.557040 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9163a1f-4478-4927-97ab-f17933671141 admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:09.557660 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9163a1f-4478-4927-97ab-f17933671141 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:09.559582 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9163a1f-4478-4927-97ab-f17933671141 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:09.561001 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cead8bee-2ea6-4d2e-9cbf-989e70b5af34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:09.561585 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cead8bee-2ea6-4d2e-9cbf-989e70b5af34 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:09.561864 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cead8bee-2ea6-4d2e-9cbf-989e70b5af34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:09.562538 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cead8bee-2ea6-4d2e-9cbf-989e70b5af34 admin admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:09.563608 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cead8bee-2ea6-4d2e-9cbf-989e70b5af34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:09.566303 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cead8bee-2ea6-4d2e-9cbf-989e70b5af34 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:09.590343 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1033/2046] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 9 12:14:09 2022] POST /identity/v3/projects => generated 389 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:14:09.594431 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9163a1f-4478-4927-97ab-f17933671141 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:09.594561 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1014/2047] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 9 12:14:09 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:14:09.598713 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cead8bee-2ea6-4d2e-9cbf-989e70b5af34 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:09.604315 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:14:09.673679 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9163a1f-4478-4927-97ab-f17933671141 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:09.676804 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cead8bee-2ea6-4d2e-9cbf-989e70b5af34 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:09.679867 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c52a45-a463-43ef-94ca-645521ed6217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:09.680949 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c52a45-a463-43ef-94ca-645521ed6217 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:09.681227 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c52a45-a463-43ef-94ca-645521ed6217 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:09.681497 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c52a45-a463-43ef-94ca-645521ed6217 admin admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:09.682085 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806186b7-011e-4917-bedf-e783c8873ab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:09.682220 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c52a45-a463-43ef-94ca-645521ed6217 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:09.682701 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806186b7-011e-4917-bedf-e783c8873ab4 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:09.683014 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806186b7-011e-4917-bedf-e783c8873ab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:09.683294 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806186b7-011e-4917-bedf-e783c8873ab4 admin admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:09.683930 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806186b7-011e-4917-bedf-e783c8873ab4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:09.684103 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c52a45-a463-43ef-94ca-645521ed6217 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:09.686166 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806186b7-011e-4917-bedf-e783c8873ab4 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:09.719701 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1034/2048] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:14:09 2022] POST /identity/v3/users => generated 295 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:14:09.722758 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1015/2049] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:14:09 2022] POST /identity/v3/users => generated 287 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:14:09.723629 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c52a45-a463-43ef-94ca-645521ed6217 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:09.723966 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806186b7-011e-4917-bedf-e783c8873ab4 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:09.734402 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c52a45-a463-43ef-94ca-645521ed6217 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:09.735138 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806186b7-011e-4917-bedf-e783c8873ab4 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:09.739504 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfcb839-3596-4019-93d1-61ea72dc303f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:09.740032 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d650bd49-001a-4830-b82f-5dd65ecedf2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:09.740173 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfcb839-3596-4019-93d1-61ea72dc303f admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:09.740371 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfcb839-3596-4019-93d1-61ea72dc303f admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:09.740624 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d650bd49-001a-4830-b82f-5dd65ecedf2c admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:09.740732 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfcb839-3596-4019-93d1-61ea72dc303f admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:09.740929 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d650bd49-001a-4830-b82f-5dd65ecedf2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:09.741170 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d650bd49-001a-4830-b82f-5dd65ecedf2c admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:09.741264 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfcb839-3596-4019-93d1-61ea72dc303f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:09.741808 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d650bd49-001a-4830-b82f-5dd65ecedf2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:09.742939 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfcb839-3596-4019-93d1-61ea72dc303f admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:09.743653 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d650bd49-001a-4830-b82f-5dd65ecedf2c admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:09.753461 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1035/2050] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:14:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 12:14:09.756848 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1016/2051] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:14:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 12:14:09.757285 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfcb839-3596-4019-93d1-61ea72dc303f admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:09.757824 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d650bd49-001a-4830-b82f-5dd65ecedf2c admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:09.767964 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfcb839-3596-4019-93d1-61ea72dc303f admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:09.768177 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d650bd49-001a-4830-b82f-5dd65ecedf2c admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:09.772470 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c8573f-4c87-4312-b939-3e632c6209d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:09.773100 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c8573f-4c87-4312-b939-3e632c6209d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:09.773376 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c8573f-4c87-4312-b939-3e632c6209d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:09.773473 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7e13c3-e742-41f7-83f4-8723a6f9ae0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:09.773648 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c8573f-4c87-4312-b939-3e632c6209d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:09.773968 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7e13c3-e742-41f7-83f4-8723a6f9ae0d admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:09.775008 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7e13c3-e742-41f7-83f4-8723a6f9ae0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:09.775130 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c8573f-4c87-4312-b939-3e632c6209d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:09.775311 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7e13c3-e742-41f7-83f4-8723a6f9ae0d admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:09.776343 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7e13c3-e742-41f7-83f4-8723a6f9ae0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:09.776988 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c8573f-4c87-4312-b939-3e632c6209d7 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:09.777939 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7e13c3-e742-41f7-83f4-8723a6f9ae0d admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:09.788532 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1017/2053] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:14:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:14:09.788772 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1036/2053] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:14:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:14:09.793373 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7e13c3-e742-41f7-83f4-8723a6f9ae0d admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:09.793499 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c8573f-4c87-4312-b939-3e632c6209d7 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:09.803294 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c8573f-4c87-4312-b939-3e632c6209d7 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:09.806819 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7e13c3-e742-41f7-83f4-8723a6f9ae0d admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:09.808367 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ed090f-975d-45ee-a82b-a50b37b2e738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:09.809055 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ed090f-975d-45ee-a82b-a50b37b2e738 admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:09.809441 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ed090f-975d-45ee-a82b-a50b37b2e738 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:09.809713 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ed090f-975d-45ee-a82b-a50b37b2e738 admin admin] PATH_INFO: `/v3/projects/aeb1faf788974a88a3333fe592e09320/users/a25d9bd4e0ce4f388b166f30ba9569bb/roles/eecd2f160dd844f5906f3d345780c501` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:09.812336 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8e4be3-3be4-4cc6-9777-46f8cb874575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:09.812995 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8e4be3-3be4-4cc6-9777-46f8cb874575 admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:09.813569 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8e4be3-3be4-4cc6-9777-46f8cb874575 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:09.813721 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8e4be3-3be4-4cc6-9777-46f8cb874575 admin admin] PATH_INFO: `/v3/projects/b4eabdd7ac6b4e6983c44fb0f6f031b1/users/b102681c7ca44cefbe8b1bdd34bbd0fb/roles/eecd2f160dd844f5906f3d345780c501` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:09.828886 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ed090f-975d-45ee-a82b-a50b37b2e738 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aeb1faf788974a88a3333fe592e09320, user_id=a25d9bd4e0ce4f388b166f30ba9569bb, role_id=eecd2f160dd844f5906f3d345780c501)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:09.832407 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ed090f-975d-45ee-a82b-a50b37b2e738 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:09.833483 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8e4be3-3be4-4cc6-9777-46f8cb874575 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4eabdd7ac6b4e6983c44fb0f6f031b1, user_id=b102681c7ca44cefbe8b1bdd34bbd0fb, role_id=eecd2f160dd844f5906f3d345780c501)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:09.836253 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8e4be3-3be4-4cc6-9777-46f8cb874575 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:09.851945 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1037/2054] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:14:09 2022] PUT /identity/v3/projects/aeb1faf788974a88a3333fe592e09320/users/a25d9bd4e0ce4f388b166f30ba9569bb/roles/eecd2f160dd844f5906f3d345780c501 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:14:09.852217 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1018/2055] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:14:09 2022] PUT /identity/v3/projects/b4eabdd7ac6b4e6983c44fb0f6f031b1/users/b102681c7ca44cefbe8b1bdd34bbd0fb/roles/eecd2f160dd844f5906f3d345780c501 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:14:09.865181 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc75c7b-1f09-4a07-b3bf-96f73acbe8f3 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:09.865476 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc75c7b-1f09-4a07-b3bf-96f73acbe8f3 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:09.865754 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc75c7b-1f09-4a07-b3bf-96f73acbe8f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:09.868839 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c4f304-fd67-4926-95bd-7fa91ad4acdb None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:09.869125 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c4f304-fd67-4926-95bd-7fa91ad4acdb None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:09.869399 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c4f304-fd67-4926-95bd-7fa91ad4acdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:09.900084 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-2bc75c7b-1f09-4a07-b3bf-96f73acbe8f3 None None] MFA Rules not processed for user `b102681c7ca44cefbe8b1bdd34bbd0fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:14:09.901342 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-2bc75c7b-1f09-4a07-b3bf-96f73acbe8f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:09.904132 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-f4c4f304-fd67-4926-95bd-7fa91ad4acdb None None] MFA Rules not processed for user `a25d9bd4e0ce4f388b166f30ba9569bb`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:14:09.905416 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-f4c4f304-fd67-4926-95bd-7fa91ad4acdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:09.951610 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:14:09.980294 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1019/2056] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:14:09 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:14:09.989618 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1038/2057] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:14:09 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:14:09.992126 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c4f304-fd67-4926-95bd-7fa91ad4acdb None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:09.994075 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-f4c4f304-fd67-4926-95bd-7fa91ad4acdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:09.998687 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc75c7b-1f09-4a07-b3bf-96f73acbe8f3 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:10.000378 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-2bc75c7b-1f09-4a07-b3bf-96f73acbe8f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:10.038354 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:14:10.058229 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc75c7b-1f09-4a07-b3bf-96f73acbe8f3 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:10.062189 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efc7ec1-edac-4171-b971-7b23ca93f85e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535c258100df40b58a5ee44b2c21d621', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:10.062810 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efc7ec1-edac-4171-b971-7b23ca93f85e service cinder] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:10.063117 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efc7ec1-edac-4171-b971-7b23ca93f85e service cinder] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:10.063386 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efc7ec1-edac-4171-b971-7b23ca93f85e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:10.072096 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efc7ec1-edac-4171-b971-7b23ca93f85e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:10.073520 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efc7ec1-edac-4171-b971-7b23ca93f85e service cinder] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:10.075626 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c4f304-fd67-4926-95bd-7fa91ad4acdb None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:10.081231 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34651cba-c690-4a2e-90c4-6f942dea651e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70bac169d5cd43e1926f310344ac11fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:10.081908 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34651cba-c690-4a2e-90c4-6f942dea651e service nova] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:10.082183 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34651cba-c690-4a2e-90c4-6f942dea651e service nova] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:10.082307 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1039/2058] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:14:09 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:14:10.083128 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34651cba-c690-4a2e-90c4-6f942dea651e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:10.091815 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34651cba-c690-4a2e-90c4-6f942dea651e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:10.093403 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34651cba-c690-4a2e-90c4-6f942dea651e service nova] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:10.103060 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1020/2059] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:14:09 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:14:10.325890 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efc7ec1-edac-4171-b971-7b23ca93f85e service cinder] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:10.403636 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efc7ec1-edac-4171-b971-7b23ca93f85e service cinder] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:10.409108 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109afb55-73da-4b84-a0c3-f75c7c1c1839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:10.409747 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109afb55-73da-4b84-a0c3-f75c7c1c1839 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:10.410023 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109afb55-73da-4b84-a0c3-f75c7c1c1839 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:10.410314 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109afb55-73da-4b84-a0c3-f75c7c1c1839 admin admin] PATH_INFO: `/v3/users/a25d9bd4e0ce4f388b166f30ba9569bb` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:10.411490 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109afb55-73da-4b84-a0c3-f75c7c1c1839 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a25d9bd4e0ce4f388b166f30ba9569bb)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:10.413413 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109afb55-73da-4b84-a0c3-f75c7c1c1839 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:10.470275 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-109afb55-73da-4b84-a0c3-f75c7c1c1839 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a25d9bd4e0ce4f388b166f30ba9569bb'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:10.478551 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-109afb55-73da-4b84-a0c3-f75c7c1c1839 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a25d9bd4e0ce4f388b166f30ba9569bb'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:10.490650 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-109afb55-73da-4b84-a0c3-f75c7c1c1839 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a25d9bd4e0ce4f388b166f30ba9569bb'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:10.499075 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-109afb55-73da-4b84-a0c3-f75c7c1c1839 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a25d9bd4e0ce4f388b166f30ba9569bb'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:10.499703 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-109afb55-73da-4b84-a0c3-f75c7c1c1839 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a25d9bd4e0ce4f388b166f30ba9569bb'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:10.501807 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:14:10.506700 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1040/2060] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 9 12:14:10 2022] DELETE /identity/v3/users/a25d9bd4e0ce4f388b166f30ba9569bb => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:14:10.511944 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cbd3e8-fe3e-452f-bc03-d0995a39db5b None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:10.512259 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cbd3e8-fe3e-452f-bc03-d0995a39db5b None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:10.512630 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cbd3e8-fe3e-452f-bc03-d0995a39db5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:10.531099 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-30cbd3e8-fe3e-452f-bc03-d0995a39db5b None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:14:10.532282 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-30cbd3e8-fe3e-452f-bc03-d0995a39db5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:10.591609 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1021/2061] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:14:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:14:10.601067 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109afb55-73da-4b84-a0c3-f75c7c1c1839 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:10.602573 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-109afb55-73da-4b84-a0c3-f75c7c1c1839 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:10.659798 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109afb55-73da-4b84-a0c3-f75c7c1c1839 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:10.663427 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c1fe07-94d1-4b17-99d9-7a91ea6bcd72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:10.664020 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c1fe07-94d1-4b17-99d9-7a91ea6bcd72 service neutron] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:10.664295 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c1fe07-94d1-4b17-99d9-7a91ea6bcd72 service neutron] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:10.664571 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c1fe07-94d1-4b17-99d9-7a91ea6bcd72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:10.673761 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c1fe07-94d1-4b17-99d9-7a91ea6bcd72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:10.675291 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c1fe07-94d1-4b17-99d9-7a91ea6bcd72 service neutron] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:10.685459 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1041/2062] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:14:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:14:10.952976 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cbd3e8-fe3e-452f-bc03-d0995a39db5b None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:10.954491 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-30cbd3e8-fe3e-452f-bc03-d0995a39db5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:10.966670 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cbd3e8-fe3e-452f-bc03-d0995a39db5b None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:10.971277 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47466018-b0a7-4ca4-b7c4-635b72de5a6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:10.971914 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47466018-b0a7-4ca4-b7c4-635b72de5a6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:10.972191 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47466018-b0a7-4ca4-b7c4-635b72de5a6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:10.972461 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47466018-b0a7-4ca4-b7c4-635b72de5a6f admin admin] PATH_INFO: `/v3/projects/aeb1faf788974a88a3333fe592e09320` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:10.973697 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47466018-b0a7-4ca4-b7c4-635b72de5a6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=aeb1faf788974a88a3333fe592e09320)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:10.975657 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47466018-b0a7-4ca4-b7c4-635b72de5a6f admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:11.025390 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-47466018-b0a7-4ca4-b7c4-635b72de5a6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aeb1faf788974a88a3333fe592e09320', 'request_id': 'req-47466018-b0a7-4ca4-b7c4-635b72de5a6f'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:11.031193 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-47466018-b0a7-4ca4-b7c4-635b72de5a6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aeb1faf788974a88a3333fe592e09320', 'request_id': 'req-47466018-b0a7-4ca4-b7c4-635b72de5a6f'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:11.039330 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-47466018-b0a7-4ca4-b7c4-635b72de5a6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aeb1faf788974a88a3333fe592e09320', 'request_id': 'req-47466018-b0a7-4ca4-b7c4-635b72de5a6f'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:11.051260 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-47466018-b0a7-4ca4-b7c4-635b72de5a6f admin admin] The token cache is being invalidate because project aeb1faf788974a88a3333fe592e09320 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:14:11.051544 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-47466018-b0a7-4ca4-b7c4-635b72de5a6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:11.052175 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-47466018-b0a7-4ca4-b7c4-635b72de5a6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:11.053735 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1022/2063] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 9 12:14:10 2022] DELETE /identity/v3/projects/aeb1faf788974a88a3333fe592e09320 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:14:11.058613 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9305f48-471d-4ec0-885a-4e390e1bbde6 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:11.058894 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9305f48-471d-4ec0-885a-4e390e1bbde6 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:11.059192 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9305f48-471d-4ec0-885a-4e390e1bbde6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:11.078560 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-d9305f48-471d-4ec0-885a-4e390e1bbde6 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:14:11.079812 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-d9305f48-471d-4ec0-885a-4e390e1bbde6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:11.139168 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1042/2064] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:14:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:14:11.148003 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9566677-cf0f-4611-92a8-05a7773ad4a9 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:11.148277 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9566677-cf0f-4611-92a8-05a7773ad4a9 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:11.148546 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9566677-cf0f-4611-92a8-05a7773ad4a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:11.170392 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-e9566677-cf0f-4611-92a8-05a7773ad4a9 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:14:11.171658 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-e9566677-cf0f-4611-92a8-05a7773ad4a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:11.179858 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1023/2065] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:14:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:14:11.184175 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9305f48-471d-4ec0-885a-4e390e1bbde6 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:11.194168 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9305f48-471d-4ec0-885a-4e390e1bbde6 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:11.199011 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafb72aa-ac58-40bf-99a4-b60957a99048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:11.199607 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafb72aa-ac58-40bf-99a4-b60957a99048 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:11.199883 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafb72aa-ac58-40bf-99a4-b60957a99048 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:11.200176 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafb72aa-ac58-40bf-99a4-b60957a99048 admin admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:11.200817 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafb72aa-ac58-40bf-99a4-b60957a99048 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:14:11.201113 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafb72aa-ac58-40bf-99a4-b60957a99048 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:11.202690 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafb72aa-ac58-40bf-99a4-b60957a99048 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:11.214796 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1043/2066] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 9 12:14:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:14:11.219007 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9566677-cf0f-4611-92a8-05a7773ad4a9 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:11.229219 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9566677-cf0f-4611-92a8-05a7773ad4a9 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:11.233929 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039c3483-95cd-4ff9-96aa-058b5e8880b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:11.234529 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039c3483-95cd-4ff9-96aa-058b5e8880b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:11.234810 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039c3483-95cd-4ff9-96aa-058b5e8880b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:11.235111 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039c3483-95cd-4ff9-96aa-058b5e8880b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:11.235748 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039c3483-95cd-4ff9-96aa-058b5e8880b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:11.237719 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039c3483-95cd-4ff9-96aa-058b5e8880b1 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:11.264362 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1024/2067] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 9 12:14:11 2022] POST /identity/v3/projects => generated 379 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:14:11.268319 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafb72aa-ac58-40bf-99a4-b60957a99048 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:11.278838 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafb72aa-ac58-40bf-99a4-b60957a99048 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:11.335990 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795f2e69-8ae6-4a6c-93c0-49f665949827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:11.336596 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795f2e69-8ae6-4a6c-93c0-49f665949827 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:11.337522 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795f2e69-8ae6-4a6c-93c0-49f665949827 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:11.337800 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795f2e69-8ae6-4a6c-93c0-49f665949827 admin admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:11.338444 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795f2e69-8ae6-4a6c-93c0-49f665949827 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:11.340716 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795f2e69-8ae6-4a6c-93c0-49f665949827 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:11.362951 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1044/2068] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:14:11 2022] POST /identity/v3/users => generated 290 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:14:11.366739 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039c3483-95cd-4ff9-96aa-058b5e8880b1 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:11.377425 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039c3483-95cd-4ff9-96aa-058b5e8880b1 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:11.381999 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dfd249-e085-4ea7-8a9b-6b48d09cd944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:11.382597 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dfd249-e085-4ea7-8a9b-6b48d09cd944 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:11.382874 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dfd249-e085-4ea7-8a9b-6b48d09cd944 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:11.383170 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dfd249-e085-4ea7-8a9b-6b48d09cd944 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:11.383855 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dfd249-e085-4ea7-8a9b-6b48d09cd944 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:11.385492 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dfd249-e085-4ea7-8a9b-6b48d09cd944 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:11.395567 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1025/2069] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:14:11 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:14:11.400025 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795f2e69-8ae6-4a6c-93c0-49f665949827 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:11.410519 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795f2e69-8ae6-4a6c-93c0-49f665949827 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:11.416097 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d01c8e-cea6-451c-bbe5-284dcd147298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:11.416940 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d01c8e-cea6-451c-bbe5-284dcd147298 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:11.417201 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d01c8e-cea6-451c-bbe5-284dcd147298 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:11.417477 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d01c8e-cea6-451c-bbe5-284dcd147298 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:11.418153 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d01c8e-cea6-451c-bbe5-284dcd147298 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:11.419769 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d01c8e-cea6-451c-bbe5-284dcd147298 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:11.430693 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1045/2070] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:14:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:14:11.435171 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dfd249-e085-4ea7-8a9b-6b48d09cd944 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:11.441666 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:14:11.445645 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dfd249-e085-4ea7-8a9b-6b48d09cd944 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:11.452415 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5c5983-3bad-4ffd-8908-8d523a93d7b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:11.453294 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5c5983-3bad-4ffd-8908-8d523a93d7b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:11.453738 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5c5983-3bad-4ffd-8908-8d523a93d7b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:11.454120 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5c5983-3bad-4ffd-8908-8d523a93d7b9 admin admin] PATH_INFO: `/v3/projects/68017f16c1d1497994236d43c501d858/users/8a21513b83e44f3d8625e020e4f9455b/roles/eecd2f160dd844f5906f3d345780c501` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:11.470788 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5c5983-3bad-4ffd-8908-8d523a93d7b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68017f16c1d1497994236d43c501d858, user_id=8a21513b83e44f3d8625e020e4f9455b, role_id=eecd2f160dd844f5906f3d345780c501)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:11.473497 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5c5983-3bad-4ffd-8908-8d523a93d7b9 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:11.488578 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1026/2071] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:14:11 2022] PUT /identity/v3/projects/68017f16c1d1497994236d43c501d858/users/8a21513b83e44f3d8625e020e4f9455b/roles/eecd2f160dd844f5906f3d345780c501 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:14:11.499623 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ab905a-b72c-4c49-a1cb-3cc19f8ffd0b None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:11.499912 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ab905a-b72c-4c49-a1cb-3cc19f8ffd0b None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:11.500184 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ab905a-b72c-4c49-a1cb-3cc19f8ffd0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:11.529606 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-c2ab905a-b72c-4c49-a1cb-3cc19f8ffd0b None None] MFA Rules not processed for user `8a21513b83e44f3d8625e020e4f9455b`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:14:11.530516 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:14:11.531564 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-c2ab905a-b72c-4c49-a1cb-3cc19f8ffd0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:11.598753 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1046/2072] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:14:11 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:14:11.603322 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5c5983-3bad-4ffd-8908-8d523a93d7b9 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:11.668101 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5c5983-3bad-4ffd-8908-8d523a93d7b9 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:11.673520 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db794af-c8c4-4a9b-bdb6-d8ab5e0d3703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:11.674217 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db794af-c8c4-4a9b-bdb6-d8ab5e0d3703 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:11.674559 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db794af-c8c4-4a9b-bdb6-d8ab5e0d3703 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:11.674892 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db794af-c8c4-4a9b-bdb6-d8ab5e0d3703 admin admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:11.675551 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db794af-c8c4-4a9b-bdb6-d8ab5e0d3703 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:11.677497 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db794af-c8c4-4a9b-bdb6-d8ab5e0d3703 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:11.705895 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1027/2073] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 9 12:14:11 2022] POST /identity/v3/projects => generated 379 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:14:11.709986 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ab905a-b72c-4c49-a1cb-3cc19f8ffd0b None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:11.721886 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ab905a-b72c-4c49-a1cb-3cc19f8ffd0b None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:11.804806 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabc8760-d1b1-4e71-9642-55ca2631b071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:11.806011 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabc8760-d1b1-4e71-9642-55ca2631b071 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:11.806318 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabc8760-d1b1-4e71-9642-55ca2631b071 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:11.806581 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabc8760-d1b1-4e71-9642-55ca2631b071 admin admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:11.807997 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabc8760-d1b1-4e71-9642-55ca2631b071 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:11.810394 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabc8760-d1b1-4e71-9642-55ca2631b071 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:11.841435 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1047/2074] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:14:11 2022] POST /identity/v3/users => generated 296 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:14:11.845830 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db794af-c8c4-4a9b-bdb6-d8ab5e0d3703 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:11.862764 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db794af-c8c4-4a9b-bdb6-d8ab5e0d3703 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:11.876515 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead92163-f08f-40e7-8cfa-1ac9ede14a31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:11.877541 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead92163-f08f-40e7-8cfa-1ac9ede14a31 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:11.877881 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead92163-f08f-40e7-8cfa-1ac9ede14a31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:11.878220 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead92163-f08f-40e7-8cfa-1ac9ede14a31 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:11.879137 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead92163-f08f-40e7-8cfa-1ac9ede14a31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:11.881888 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead92163-f08f-40e7-8cfa-1ac9ede14a31 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:11.895540 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1028/2075] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:14:11 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 12:14:11.904335 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabc8760-d1b1-4e71-9642-55ca2631b071 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:11.914492 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabc8760-d1b1-4e71-9642-55ca2631b071 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:11.919849 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c367d94-aa2d-4a1c-ae41-2e6dd3e9a6f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:11.920520 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c367d94-aa2d-4a1c-ae41-2e6dd3e9a6f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:11.920798 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c367d94-aa2d-4a1c-ae41-2e6dd3e9a6f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:11.921072 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c367d94-aa2d-4a1c-ae41-2e6dd3e9a6f9 admin admin] PATH_INFO: `/v3/projects/3d1554bc519b4fe4a2c28035459bc4b1/users/8d5e677529ae447791c6f132fedbe837/roles/3f2ca33902a04b6dbc2ef7d06d7289d3` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:11.937250 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c367d94-aa2d-4a1c-ae41-2e6dd3e9a6f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d1554bc519b4fe4a2c28035459bc4b1, user_id=8d5e677529ae447791c6f132fedbe837, role_id=3f2ca33902a04b6dbc2ef7d06d7289d3)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:11.939999 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c367d94-aa2d-4a1c-ae41-2e6dd3e9a6f9 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:11.954427 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1048/2076] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:14:11 2022] PUT /identity/v3/projects/3d1554bc519b4fe4a2c28035459bc4b1/users/8d5e677529ae447791c6f132fedbe837/roles/3f2ca33902a04b6dbc2ef7d06d7289d3 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:14:11.958387 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead92163-f08f-40e7-8cfa-1ac9ede14a31 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:11.968744 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead92163-f08f-40e7-8cfa-1ac9ede14a31 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:12.034720 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dfde58-5060-4a84-8ac1-b4ec362454db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:12.035377 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dfde58-5060-4a84-8ac1-b4ec362454db admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:12.035660 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dfde58-5060-4a84-8ac1-b4ec362454db admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:12.035962 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dfde58-5060-4a84-8ac1-b4ec362454db admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:12.036575 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dfde58-5060-4a84-8ac1-b4ec362454db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:12.038423 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dfde58-5060-4a84-8ac1-b4ec362454db admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:12.048772 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1029/2077] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:14:11 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:14:12.054924 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c367d94-aa2d-4a1c-ae41-2e6dd3e9a6f9 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:12.065405 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c367d94-aa2d-4a1c-ae41-2e6dd3e9a6f9 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:12.070271 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6d7613-a67a-41ae-96a4-1bd47ca511dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:12.070960 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6d7613-a67a-41ae-96a4-1bd47ca511dd admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:12.071948 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6d7613-a67a-41ae-96a4-1bd47ca511dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:12.072328 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6d7613-a67a-41ae-96a4-1bd47ca511dd admin admin] PATH_INFO: `/v3/projects/3d1554bc519b4fe4a2c28035459bc4b1/users/8d5e677529ae447791c6f132fedbe837/roles/3f2ca33902a04b6dbc2ef7d06d7289d3` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:12.074258 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6d7613-a67a-41ae-96a4-1bd47ca511dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d1554bc519b4fe4a2c28035459bc4b1, user_id=8d5e677529ae447791c6f132fedbe837, role_id=3f2ca33902a04b6dbc2ef7d06d7289d3)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:12.076917 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6d7613-a67a-41ae-96a4-1bd47ca511dd admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:12.087496 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1049/2078] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:14:12 2022] PUT /identity/v3/projects/3d1554bc519b4fe4a2c28035459bc4b1/users/8d5e677529ae447791c6f132fedbe837/roles/3f2ca33902a04b6dbc2ef7d06d7289d3 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:14:12.097623 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2993cb4-90ad-48b7-9915-5a1dea79f0a2 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:12.097908 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2993cb4-90ad-48b7-9915-5a1dea79f0a2 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:12.098204 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2993cb4-90ad-48b7-9915-5a1dea79f0a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:12.126985 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-b2993cb4-90ad-48b7-9915-5a1dea79f0a2 None None] MFA Rules not processed for user `8d5e677529ae447791c6f132fedbe837`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:14:12.128181 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-b2993cb4-90ad-48b7-9915-5a1dea79f0a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:12.169029 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:14:12.200380 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1030/2079] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:14:12 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:14:12.213260 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6d7613-a67a-41ae-96a4-1bd47ca511dd admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:12.215161 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-ab6d7613-a67a-41ae-96a4-1bd47ca511dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:12.279978 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6d7613-a67a-41ae-96a4-1bd47ca511dd admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:12.283694 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cb425f-863f-4981-bb76-cf2293fdb8db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535c258100df40b58a5ee44b2c21d621', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:12.284302 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cb425f-863f-4981-bb76-cf2293fdb8db service cinder] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:12.284576 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cb425f-863f-4981-bb76-cf2293fdb8db service cinder] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:12.284842 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cb425f-863f-4981-bb76-cf2293fdb8db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:12.291516 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:14:12.296566 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cb425f-863f-4981-bb76-cf2293fdb8db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:12.298376 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cb425f-863f-4981-bb76-cf2293fdb8db service cinder] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:12.309623 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1050/2080] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:14:12 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:14:12.388323 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2993cb4-90ad-48b7-9915-5a1dea79f0a2 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:12.397683 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2993cb4-90ad-48b7-9915-5a1dea79f0a2 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:12.401730 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a8de38-27a0-4533-9a9b-89876825272a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535c258100df40b58a5ee44b2c21d621', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:12.402325 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a8de38-27a0-4533-9a9b-89876825272a service cinder] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:12.402604 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a8de38-27a0-4533-9a9b-89876825272a service cinder] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:12.402877 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a8de38-27a0-4533-9a9b-89876825272a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:12.459604 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a8de38-27a0-4533-9a9b-89876825272a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:12.460972 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a8de38-27a0-4533-9a9b-89876825272a service cinder] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:12.470387 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1031/2081] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:14:12 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:14:22.304274 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=3 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:14:22.466004 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=39 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:14:24.376755 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cb425f-863f-4981-bb76-cf2293fdb8db service cinder] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:24.378156 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-58cb425f-863f-4981-bb76-cf2293fdb8db service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:24.442665 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cb425f-863f-4981-bb76-cf2293fdb8db service cinder] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:24.448365 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d25cc89-a610-487b-b45f-ebe5e8d6fdc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:24.449042 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d25cc89-a610-487b-b45f-ebe5e8d6fdc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:24.449374 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d25cc89-a610-487b-b45f-ebe5e8d6fdc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:24.449561 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d25cc89-a610-487b-b45f-ebe5e8d6fdc4 admin admin] PATH_INFO: `/v3/users/1f5cd4165918469d978d269db2b46599` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:24.451019 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d25cc89-a610-487b-b45f-ebe5e8d6fdc4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f5cd4165918469d978d269db2b46599)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:24.452970 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d25cc89-a610-487b-b45f-ebe5e8d6fdc4 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:24.502062 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-1d25cc89-a610-487b-b45f-ebe5e8d6fdc4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f5cd4165918469d978d269db2b46599'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:24.510442 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-1d25cc89-a610-487b-b45f-ebe5e8d6fdc4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f5cd4165918469d978d269db2b46599'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:24.523160 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-1d25cc89-a610-487b-b45f-ebe5e8d6fdc4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f5cd4165918469d978d269db2b46599'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:24.534479 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-1d25cc89-a610-487b-b45f-ebe5e8d6fdc4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f5cd4165918469d978d269db2b46599'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:24.535433 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-1d25cc89-a610-487b-b45f-ebe5e8d6fdc4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f5cd4165918469d978d269db2b46599'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:24.542180 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1051/2082] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 9 12:14:24 2022] DELETE /identity/v3/users/1f5cd4165918469d978d269db2b46599 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:14:24.546727 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2863e010-15c9-43ef-bee4-7e98dc4c6e39 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:24.547252 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2863e010-15c9-43ef-bee4-7e98dc4c6e39 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:24.547358 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2863e010-15c9-43ef-bee4-7e98dc4c6e39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:24.568158 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-2863e010-15c9-43ef-bee4-7e98dc4c6e39 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:14:24.569982 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-2863e010-15c9-43ef-bee4-7e98dc4c6e39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:24.635636 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1032/2083] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:14:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:14:24.647643 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d25cc89-a610-487b-b45f-ebe5e8d6fdc4 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:24.649469 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-1d25cc89-a610-487b-b45f-ebe5e8d6fdc4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:24.713617 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d25cc89-a610-487b-b45f-ebe5e8d6fdc4 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:24.717222 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7489557d-6d93-4c54-9421-3f5f972f06fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:24.718413 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7489557d-6d93-4c54-9421-3f5f972f06fc service neutron] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:24.718698 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7489557d-6d93-4c54-9421-3f5f972f06fc service neutron] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:24.719001 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7489557d-6d93-4c54-9421-3f5f972f06fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:24.728235 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7489557d-6d93-4c54-9421-3f5f972f06fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:24.729682 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7489557d-6d93-4c54-9421-3f5f972f06fc service neutron] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:24.739093 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1052/2084] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:14:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:14:25.206236 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2863e010-15c9-43ef-bee4-7e98dc4c6e39 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:25.207719 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-2863e010-15c9-43ef-bee4-7e98dc4c6e39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:25.222121 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2863e010-15c9-43ef-bee4-7e98dc4c6e39 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:25.228619 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1389fc78-bd60-41bc-90ba-c3d27814faf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:25.229472 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1389fc78-bd60-41bc-90ba-c3d27814faf2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:25.229552 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1389fc78-bd60-41bc-90ba-c3d27814faf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:25.229815 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1389fc78-bd60-41bc-90ba-c3d27814faf2 admin admin] PATH_INFO: `/v3/projects/5c082fded13e4c5399f3ce0d6778e7d8` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:25.230942 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1389fc78-bd60-41bc-90ba-c3d27814faf2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c082fded13e4c5399f3ce0d6778e7d8)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:25.232841 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1389fc78-bd60-41bc-90ba-c3d27814faf2 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:25.279834 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-1389fc78-bd60-41bc-90ba-c3d27814faf2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c082fded13e4c5399f3ce0d6778e7d8', 'request_id': 'req-1389fc78-bd60-41bc-90ba-c3d27814faf2'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:25.285348 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-1389fc78-bd60-41bc-90ba-c3d27814faf2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c082fded13e4c5399f3ce0d6778e7d8', 'request_id': 'req-1389fc78-bd60-41bc-90ba-c3d27814faf2'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:25.293141 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-1389fc78-bd60-41bc-90ba-c3d27814faf2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c082fded13e4c5399f3ce0d6778e7d8', 'request_id': 'req-1389fc78-bd60-41bc-90ba-c3d27814faf2'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:25.304195 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-1389fc78-bd60-41bc-90ba-c3d27814faf2 admin admin] The token cache is being invalidate because project 5c082fded13e4c5399f3ce0d6778e7d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:14:25.304484 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-1389fc78-bd60-41bc-90ba-c3d27814faf2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:25.305814 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-1389fc78-bd60-41bc-90ba-c3d27814faf2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:25.307322 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1033/2085] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 9 12:14:25 2022] DELETE /identity/v3/projects/5c082fded13e4c5399f3ce0d6778e7d8 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:14:25.311557 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7489557d-6d93-4c54-9421-3f5f972f06fc service neutron] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:25.321095 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7489557d-6d93-4c54-9421-3f5f972f06fc service neutron] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:25.339552 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:14:25.382161 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92e27f5-cf9c-4af8-8327-efe8af43945d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:25.382777 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92e27f5-cf9c-4af8-8327-efe8af43945d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:25.383301 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92e27f5-cf9c-4af8-8327-efe8af43945d admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:25.383580 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92e27f5-cf9c-4af8-8327-efe8af43945d admin admin] PATH_INFO: `/v3/users/29758757a0b24cd4abf69f87ea9751f2` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:25.384701 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92e27f5-cf9c-4af8-8327-efe8af43945d admin admin] RBAC: Authorizing `identity:delete_user(user_id=29758757a0b24cd4abf69f87ea9751f2)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:25.386591 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92e27f5-cf9c-4af8-8327-efe8af43945d admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:25.432503 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-c92e27f5-cf9c-4af8-8327-efe8af43945d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29758757a0b24cd4abf69f87ea9751f2'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:25.439784 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-c92e27f5-cf9c-4af8-8327-efe8af43945d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29758757a0b24cd4abf69f87ea9751f2'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:25.452153 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-c92e27f5-cf9c-4af8-8327-efe8af43945d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29758757a0b24cd4abf69f87ea9751f2'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:25.459930 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-c92e27f5-cf9c-4af8-8327-efe8af43945d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29758757a0b24cd4abf69f87ea9751f2'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:25.460635 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-c92e27f5-cf9c-4af8-8327-efe8af43945d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29758757a0b24cd4abf69f87ea9751f2'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:25.467123 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1053/2086] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 9 12:14:25 2022] DELETE /identity/v3/users/29758757a0b24cd4abf69f87ea9751f2 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:14:25.798243 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1389fc78-bd60-41bc-90ba-c3d27814faf2 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:25.799814 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-1389fc78-bd60-41bc-90ba-c3d27814faf2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:25.813344 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1389fc78-bd60-41bc-90ba-c3d27814faf2 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:25.881147 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1ae60c-beb3-4bb3-8142-522232b11dc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:25.881774 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1ae60c-beb3-4bb3-8142-522232b11dc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:25.882062 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1ae60c-beb3-4bb3-8142-522232b11dc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:25.882335 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1ae60c-beb3-4bb3-8142-522232b11dc8 admin admin] PATH_INFO: `/v3/projects/f3d1e4d790514a198e8d838633700558` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:25.883641 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1ae60c-beb3-4bb3-8142-522232b11dc8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3d1e4d790514a198e8d838633700558)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:25.885581 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1ae60c-beb3-4bb3-8142-522232b11dc8 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:25.920887 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:14:25.940183 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-5b1ae60c-beb3-4bb3-8142-522232b11dc8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3d1e4d790514a198e8d838633700558', 'request_id': 'req-5b1ae60c-beb3-4bb3-8142-522232b11dc8'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:25.945906 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-5b1ae60c-beb3-4bb3-8142-522232b11dc8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3d1e4d790514a198e8d838633700558', 'request_id': 'req-5b1ae60c-beb3-4bb3-8142-522232b11dc8'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:25.953881 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-5b1ae60c-beb3-4bb3-8142-522232b11dc8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3d1e4d790514a198e8d838633700558', 'request_id': 'req-5b1ae60c-beb3-4bb3-8142-522232b11dc8'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:25.964819 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-5b1ae60c-beb3-4bb3-8142-522232b11dc8 admin admin] The token cache is being invalidate because project f3d1e4d790514a198e8d838633700558 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:14:25.965111 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-5b1ae60c-beb3-4bb3-8142-522232b11dc8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:25.965728 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-5b1ae60c-beb3-4bb3-8142-522232b11dc8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:25.967319 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1034/2087] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 9 12:14:25 2022] DELETE /identity/v3/projects/f3d1e4d790514a198e8d838633700558 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:14:25.972135 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b557cd-0ad6-4fd9-ae0d-d6ace3ab0e5a None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:25.972425 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b557cd-0ad6-4fd9-ae0d-d6ace3ab0e5a None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:25.972701 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b557cd-0ad6-4fd9-ae0d-d6ace3ab0e5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:25.993456 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-79b557cd-0ad6-4fd9-ae0d-d6ace3ab0e5a None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:14:25.994657 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-79b557cd-0ad6-4fd9-ae0d-d6ace3ab0e5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:26.052145 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1054/2088] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:14:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:14:26.060973 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b468f242-a85f-4008-b0e3-a35172631122 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:26.061264 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b468f242-a85f-4008-b0e3-a35172631122 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:26.061538 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b468f242-a85f-4008-b0e3-a35172631122 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:26.081380 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-b468f242-a85f-4008-b0e3-a35172631122 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:14:26.082577 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-b468f242-a85f-4008-b0e3-a35172631122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:26.091227 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1035/2089] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:14:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:14:26.095385 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b557cd-0ad6-4fd9-ae0d-d6ace3ab0e5a None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:26.105328 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b557cd-0ad6-4fd9-ae0d-d6ace3ab0e5a None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:26.109812 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a28517-65b0-48c8-9178-a2b15fee2aeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:26.110403 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a28517-65b0-48c8-9178-a2b15fee2aeb admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:26.110684 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a28517-65b0-48c8-9178-a2b15fee2aeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:26.110990 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a28517-65b0-48c8-9178-a2b15fee2aeb admin admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:26.111632 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a28517-65b0-48c8-9178-a2b15fee2aeb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:14:26.111932 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a28517-65b0-48c8-9178-a2b15fee2aeb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:26.113510 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a28517-65b0-48c8-9178-a2b15fee2aeb admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:26.124816 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1055/2090] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 9 12:14:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:14:26.128882 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b468f242-a85f-4008-b0e3-a35172631122 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:26.138174 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b468f242-a85f-4008-b0e3-a35172631122 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:26.142778 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab91254-a687-4860-b246-820d3fb15d20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:26.143400 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab91254-a687-4860-b246-820d3fb15d20 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:26.143676 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab91254-a687-4860-b246-820d3fb15d20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:26.143944 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab91254-a687-4860-b246-820d3fb15d20 admin admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:26.144573 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab91254-a687-4860-b246-820d3fb15d20 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:26.146469 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab91254-a687-4860-b246-820d3fb15d20 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:26.174443 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1036/2091] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 9 12:14:26 2022] POST /identity/v3/projects => generated 371 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:14:26.178518 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a28517-65b0-48c8-9178-a2b15fee2aeb admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:26.189678 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a28517-65b0-48c8-9178-a2b15fee2aeb admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:26.215471 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:14:26.248668 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063905bc-b6be-4bbf-b5c1-0ff8ea60a4ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:26.249272 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063905bc-b6be-4bbf-b5c1-0ff8ea60a4ac admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:26.249555 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063905bc-b6be-4bbf-b5c1-0ff8ea60a4ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:26.249827 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063905bc-b6be-4bbf-b5c1-0ff8ea60a4ac admin admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:26.250483 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063905bc-b6be-4bbf-b5c1-0ff8ea60a4ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:26.252436 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063905bc-b6be-4bbf-b5c1-0ff8ea60a4ac admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:26.276850 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1056/2092] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:14:26 2022] POST /identity/v3/users => generated 286 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:14:26.281047 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab91254-a687-4860-b246-820d3fb15d20 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:26.294619 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab91254-a687-4860-b246-820d3fb15d20 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:26.299263 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a87135d-06d0-4e0a-a34b-a367b7e0a518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:26.299896 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a87135d-06d0-4e0a-a34b-a367b7e0a518 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:26.300174 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a87135d-06d0-4e0a-a34b-a367b7e0a518 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:26.300443 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a87135d-06d0-4e0a-a34b-a367b7e0a518 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:26.301083 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a87135d-06d0-4e0a-a34b-a367b7e0a518 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:26.302682 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a87135d-06d0-4e0a-a34b-a367b7e0a518 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:26.313238 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1037/2093] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:14:26 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 12:14:26.317172 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063905bc-b6be-4bbf-b5c1-0ff8ea60a4ac admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:26.327153 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063905bc-b6be-4bbf-b5c1-0ff8ea60a4ac admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:26.331819 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa5b1a5-ac22-46f7-9099-06f8fc8581ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:26.332405 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa5b1a5-ac22-46f7-9099-06f8fc8581ed admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:26.332678 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa5b1a5-ac22-46f7-9099-06f8fc8581ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:26.332947 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa5b1a5-ac22-46f7-9099-06f8fc8581ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:26.333576 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa5b1a5-ac22-46f7-9099-06f8fc8581ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:26.335197 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa5b1a5-ac22-46f7-9099-06f8fc8581ed admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:26.346999 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1057/2094] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:14:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:14:26.350970 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a87135d-06d0-4e0a-a34b-a367b7e0a518 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:26.361726 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a87135d-06d0-4e0a-a34b-a367b7e0a518 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:26.366431 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c890bc-2835-456a-963d-196da962a5bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:26.367119 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c890bc-2835-456a-963d-196da962a5bd admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:26.367425 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c890bc-2835-456a-963d-196da962a5bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:26.367697 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c890bc-2835-456a-963d-196da962a5bd admin admin] PATH_INFO: `/v3/projects/cdab9b7a5d6645ad8b03bebdb3ca1b24/users/da1c144d866b40fd88f1dc017ee8dcb1/roles/eecd2f160dd844f5906f3d345780c501` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:26.382085 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c890bc-2835-456a-963d-196da962a5bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=cdab9b7a5d6645ad8b03bebdb3ca1b24, user_id=da1c144d866b40fd88f1dc017ee8dcb1, role_id=eecd2f160dd844f5906f3d345780c501)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:26.384749 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c890bc-2835-456a-963d-196da962a5bd admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:26.398100 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1038/2095] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:14:26 2022] PUT /identity/v3/projects/cdab9b7a5d6645ad8b03bebdb3ca1b24/users/da1c144d866b40fd88f1dc017ee8dcb1/roles/eecd2f160dd844f5906f3d345780c501 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:14:26.409232 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810e4ab4-7472-4988-8e53-1a17b3d77860 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:26.409719 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810e4ab4-7472-4988-8e53-1a17b3d77860 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:26.409839 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810e4ab4-7472-4988-8e53-1a17b3d77860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:26.438510 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-810e4ab4-7472-4988-8e53-1a17b3d77860 None None] MFA Rules not processed for user `da1c144d866b40fd88f1dc017ee8dcb1`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:14:26.439755 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-810e4ab4-7472-4988-8e53-1a17b3d77860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:26.513566 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1058/2096] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:14:26 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:14:26.517754 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c890bc-2835-456a-963d-196da962a5bd admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:26.588183 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c890bc-2835-456a-963d-196da962a5bd admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:26.595101 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e920d6b8-02c0-4e18-a88d-0883a510269a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:26.595771 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e920d6b8-02c0-4e18-a88d-0883a510269a admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:26.596108 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e920d6b8-02c0-4e18-a88d-0883a510269a admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:26.596487 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e920d6b8-02c0-4e18-a88d-0883a510269a admin admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:26.597322 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e920d6b8-02c0-4e18-a88d-0883a510269a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:26.599721 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e920d6b8-02c0-4e18-a88d-0883a510269a admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:26.629279 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1039/2097] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 9 12:14:26 2022] POST /identity/v3/projects => generated 377 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:14:26.635481 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810e4ab4-7472-4988-8e53-1a17b3d77860 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:26.648007 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810e4ab4-7472-4988-8e53-1a17b3d77860 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:26.719084 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f37b3b7-fcdd-4168-a477-4c6c5493d7ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:26.719796 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f37b3b7-fcdd-4168-a477-4c6c5493d7ba admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:26.720131 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f37b3b7-fcdd-4168-a477-4c6c5493d7ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:26.720344 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f37b3b7-fcdd-4168-a477-4c6c5493d7ba admin admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:26.720981 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f37b3b7-fcdd-4168-a477-4c6c5493d7ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:26.723484 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f37b3b7-fcdd-4168-a477-4c6c5493d7ba admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:26.746718 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1059/2098] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:14:26 2022] POST /identity/v3/users => generated 295 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:14:26.751357 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e920d6b8-02c0-4e18-a88d-0883a510269a admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:26.765013 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e920d6b8-02c0-4e18-a88d-0883a510269a admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:26.769844 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f5f5f9-66a4-4bea-8ab5-7c0ba4b9372e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:26.770649 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f5f5f9-66a4-4bea-8ab5-7c0ba4b9372e admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:26.771014 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f5f5f9-66a4-4bea-8ab5-7c0ba4b9372e admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:26.771332 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f5f5f9-66a4-4bea-8ab5-7c0ba4b9372e admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:26.771985 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f5f5f9-66a4-4bea-8ab5-7c0ba4b9372e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:26.774257 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f5f5f9-66a4-4bea-8ab5-7c0ba4b9372e admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:26.788273 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1040/2099] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:14:26 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:14:26.793245 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f37b3b7-fcdd-4168-a477-4c6c5493d7ba admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:26.803834 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f37b3b7-fcdd-4168-a477-4c6c5493d7ba admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:26.809060 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caf2eb5-689c-413a-9143-e49e9a54ff4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:26.809966 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caf2eb5-689c-413a-9143-e49e9a54ff4e admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:26.810238 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caf2eb5-689c-413a-9143-e49e9a54ff4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:26.810501 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caf2eb5-689c-413a-9143-e49e9a54ff4e admin admin] PATH_INFO: `/v3/projects/2d091d9f3287438d81ee47dee3bef467/users/fae3216365774960955aa5e31b2d3ebb/roles/3f2ca33902a04b6dbc2ef7d06d7289d3` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:26.827565 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caf2eb5-689c-413a-9143-e49e9a54ff4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d091d9f3287438d81ee47dee3bef467, user_id=fae3216365774960955aa5e31b2d3ebb, role_id=3f2ca33902a04b6dbc2ef7d06d7289d3)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:26.830310 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caf2eb5-689c-413a-9143-e49e9a54ff4e admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:26.843857 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1060/2100] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:14:26 2022] PUT /identity/v3/projects/2d091d9f3287438d81ee47dee3bef467/users/fae3216365774960955aa5e31b2d3ebb/roles/3f2ca33902a04b6dbc2ef7d06d7289d3 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:14:26.848554 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f5f5f9-66a4-4bea-8ab5-7c0ba4b9372e admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:26.860099 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f5f5f9-66a4-4bea-8ab5-7c0ba4b9372e admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:26.882047 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:14:26.927075 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05aae07d-6dc1-4b5b-9b44-7371f3c3e7bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:26.928187 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aae07d-6dc1-4b5b-9b44-7371f3c3e7bc admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:26.928187 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aae07d-6dc1-4b5b-9b44-7371f3c3e7bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:26.928187 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aae07d-6dc1-4b5b-9b44-7371f3c3e7bc admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:26.928738 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05aae07d-6dc1-4b5b-9b44-7371f3c3e7bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:26.931189 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05aae07d-6dc1-4b5b-9b44-7371f3c3e7bc admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:26.945019 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1041/2101] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:14:26 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:14:26.947155 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caf2eb5-689c-413a-9143-e49e9a54ff4e admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:26.953265 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:14:26.960386 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caf2eb5-689c-413a-9143-e49e9a54ff4e admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:26.967425 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb33872-8b5b-4fc5-83d4-03a8486d9787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:26.968257 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb33872-8b5b-4fc5-83d4-03a8486d9787 admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:26.968873 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb33872-8b5b-4fc5-83d4-03a8486d9787 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:26.969578 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb33872-8b5b-4fc5-83d4-03a8486d9787 admin admin] PATH_INFO: `/v3/projects/2d091d9f3287438d81ee47dee3bef467/users/fae3216365774960955aa5e31b2d3ebb/roles/3f2ca33902a04b6dbc2ef7d06d7289d3` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:26.971881 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb33872-8b5b-4fc5-83d4-03a8486d9787 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d091d9f3287438d81ee47dee3bef467, user_id=fae3216365774960955aa5e31b2d3ebb, role_id=3f2ca33902a04b6dbc2ef7d06d7289d3)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:26.975079 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb33872-8b5b-4fc5-83d4-03a8486d9787 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:26.990612 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1061/2102] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:14:26 2022] PUT /identity/v3/projects/2d091d9f3287438d81ee47dee3bef467/users/fae3216365774960955aa5e31b2d3ebb/roles/3f2ca33902a04b6dbc2ef7d06d7289d3 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:14:27.002582 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8443aa5b-8e9c-46ec-82ae-168abc52b03d None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:27.003513 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8443aa5b-8e9c-46ec-82ae-168abc52b03d None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:27.004447 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8443aa5b-8e9c-46ec-82ae-168abc52b03d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:27.038472 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-8443aa5b-8e9c-46ec-82ae-168abc52b03d None None] MFA Rules not processed for user `fae3216365774960955aa5e31b2d3ebb`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:14:27.039729 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-8443aa5b-8e9c-46ec-82ae-168abc52b03d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:27.124688 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1042/2103] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:14:27 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:14:27.137773 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb33872-8b5b-4fc5-83d4-03a8486d9787 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:27.139856 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-5fb33872-8b5b-4fc5-83d4-03a8486d9787 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:27.212466 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb33872-8b5b-4fc5-83d4-03a8486d9787 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:27.216182 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a117d705-cd1b-4b97-a773-03313a85e3ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535c258100df40b58a5ee44b2c21d621', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:27.216786 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a117d705-cd1b-4b97-a773-03313a85e3ad service cinder] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:27.217058 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a117d705-cd1b-4b97-a773-03313a85e3ad service cinder] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:27.217328 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a117d705-cd1b-4b97-a773-03313a85e3ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:27.277682 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a117d705-cd1b-4b97-a773-03313a85e3ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:27.279217 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a117d705-cd1b-4b97-a773-03313a85e3ad service cinder] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:27.291235 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1062/2104] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:14:27 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:14:27.708737 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8443aa5b-8e9c-46ec-82ae-168abc52b03d None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:27.710382 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-8443aa5b-8e9c-46ec-82ae-168abc52b03d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:27.778889 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8443aa5b-8e9c-46ec-82ae-168abc52b03d None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:27.784179 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487cac7a-1b3a-4510-9dd0-dac10ac25d68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:27.784880 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487cac7a-1b3a-4510-9dd0-dac10ac25d68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:27.785082 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487cac7a-1b3a-4510-9dd0-dac10ac25d68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:27.785352 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487cac7a-1b3a-4510-9dd0-dac10ac25d68 admin admin] PATH_INFO: `/v3/users/8a21513b83e44f3d8625e020e4f9455b` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:27.786476 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487cac7a-1b3a-4510-9dd0-dac10ac25d68 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a21513b83e44f3d8625e020e4f9455b)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:27.788436 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487cac7a-1b3a-4510-9dd0-dac10ac25d68 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:27.837814 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-487cac7a-1b3a-4510-9dd0-dac10ac25d68 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a21513b83e44f3d8625e020e4f9455b'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:27.839038 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:14:27.854632 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-487cac7a-1b3a-4510-9dd0-dac10ac25d68 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a21513b83e44f3d8625e020e4f9455b'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:27.863471 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-487cac7a-1b3a-4510-9dd0-dac10ac25d68 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a21513b83e44f3d8625e020e4f9455b'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:27.872089 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-487cac7a-1b3a-4510-9dd0-dac10ac25d68 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a21513b83e44f3d8625e020e4f9455b'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:27.872762 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-487cac7a-1b3a-4510-9dd0-dac10ac25d68 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a21513b83e44f3d8625e020e4f9455b'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:27.879638 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1043/2105] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 9 12:14:27 2022] DELETE /identity/v3/users/8a21513b83e44f3d8625e020e4f9455b => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:14:27.884152 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc16dd7-9cf3-4214-abf8-3860622355b7 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:27.884660 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc16dd7-9cf3-4214-abf8-3860622355b7 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:27.884762 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc16dd7-9cf3-4214-abf8-3860622355b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:27.907782 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-9cc16dd7-9cf3-4214-abf8-3860622355b7 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:14:27.908991 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-9cc16dd7-9cf3-4214-abf8-3860622355b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:27.974356 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1063/2106] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:14:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:14:27.984356 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487cac7a-1b3a-4510-9dd0-dac10ac25d68 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:27.985816 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-487cac7a-1b3a-4510-9dd0-dac10ac25d68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:28.060936 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487cac7a-1b3a-4510-9dd0-dac10ac25d68 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:28.068312 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108b2812-5a83-4bb3-810a-7e3e96a049a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:28.068921 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108b2812-5a83-4bb3-810a-7e3e96a049a0 service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:28.069198 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108b2812-5a83-4bb3-810a-7e3e96a049a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:28.069897 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108b2812-5a83-4bb3-810a-7e3e96a049a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:28.080530 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108b2812-5a83-4bb3-810a-7e3e96a049a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:28.081987 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108b2812-5a83-4bb3-810a-7e3e96a049a0 service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:28.092962 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1044/2107] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:14:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:14:28.497858 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc16dd7-9cf3-4214-abf8-3860622355b7 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:28.501572 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-9cc16dd7-9cf3-4214-abf8-3860622355b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:28.514710 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc16dd7-9cf3-4214-abf8-3860622355b7 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:28.524738 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0cd9f1-d364-4033-81b6-e0b252250c54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:28.525405 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0cd9f1-d364-4033-81b6-e0b252250c54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:28.525682 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0cd9f1-d364-4033-81b6-e0b252250c54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:28.525953 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0cd9f1-d364-4033-81b6-e0b252250c54 admin admin] PATH_INFO: `/v3/projects/68017f16c1d1497994236d43c501d858` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:28.527407 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0cd9f1-d364-4033-81b6-e0b252250c54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=68017f16c1d1497994236d43c501d858)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:28.529337 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0cd9f1-d364-4033-81b6-e0b252250c54 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:28.573544 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:14:28.582114 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-9e0cd9f1-d364-4033-81b6-e0b252250c54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68017f16c1d1497994236d43c501d858', 'request_id': 'req-9e0cd9f1-d364-4033-81b6-e0b252250c54'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:28.589595 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-9e0cd9f1-d364-4033-81b6-e0b252250c54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68017f16c1d1497994236d43c501d858', 'request_id': 'req-9e0cd9f1-d364-4033-81b6-e0b252250c54'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:28.596666 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-9e0cd9f1-d364-4033-81b6-e0b252250c54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68017f16c1d1497994236d43c501d858', 'request_id': 'req-9e0cd9f1-d364-4033-81b6-e0b252250c54'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:28.610168 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-9e0cd9f1-d364-4033-81b6-e0b252250c54 admin admin] The token cache is being invalidate because project 68017f16c1d1497994236d43c501d858 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81591) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:14:28.610463 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-9e0cd9f1-d364-4033-81b6-e0b252250c54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:28.611133 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-9e0cd9f1-d364-4033-81b6-e0b252250c54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:28.612636 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1064/2108] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 9 12:14:28 2022] DELETE /identity/v3/projects/68017f16c1d1497994236d43c501d858 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:14:28.616711 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108b2812-5a83-4bb3-810a-7e3e96a049a0 service neutron] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:28.626232 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108b2812-5a83-4bb3-810a-7e3e96a049a0 service neutron] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:28.688497 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126b3ce7-bee9-4ccd-8ccb-61ad633a1b28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:28.689502 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126b3ce7-bee9-4ccd-8ccb-61ad633a1b28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:28.689502 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126b3ce7-bee9-4ccd-8ccb-61ad633a1b28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:28.689684 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126b3ce7-bee9-4ccd-8ccb-61ad633a1b28 admin admin] PATH_INFO: `/v3/users/8d5e677529ae447791c6f132fedbe837` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:28.690794 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126b3ce7-bee9-4ccd-8ccb-61ad633a1b28 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d5e677529ae447791c6f132fedbe837)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:28.692750 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126b3ce7-bee9-4ccd-8ccb-61ad633a1b28 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:28.743589 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-126b3ce7-bee9-4ccd-8ccb-61ad633a1b28 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d5e677529ae447791c6f132fedbe837'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:28.755666 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-126b3ce7-bee9-4ccd-8ccb-61ad633a1b28 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d5e677529ae447791c6f132fedbe837'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:28.760913 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:14:28.763571 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-126b3ce7-bee9-4ccd-8ccb-61ad633a1b28 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d5e677529ae447791c6f132fedbe837'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:28.771110 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-126b3ce7-bee9-4ccd-8ccb-61ad633a1b28 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d5e677529ae447791c6f132fedbe837'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:28.771887 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-126b3ce7-bee9-4ccd-8ccb-61ad633a1b28 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d5e677529ae447791c6f132fedbe837'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:28.781770 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1045/2109] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 9 12:14:28 2022] DELETE /identity/v3/users/8d5e677529ae447791c6f132fedbe837 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:14:29.224905 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0cd9f1-d364-4033-81b6-e0b252250c54 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:29.226534 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-9e0cd9f1-d364-4033-81b6-e0b252250c54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:29.239903 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0cd9f1-d364-4033-81b6-e0b252250c54 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:29.308534 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30a6816-1e69-4135-a2fb-5b187bbdcf21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:29.309175 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30a6816-1e69-4135-a2fb-5b187bbdcf21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:29.309710 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30a6816-1e69-4135-a2fb-5b187bbdcf21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:29.309986 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30a6816-1e69-4135-a2fb-5b187bbdcf21 admin admin] PATH_INFO: `/v3/projects/3d1554bc519b4fe4a2c28035459bc4b1` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:29.311139 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30a6816-1e69-4135-a2fb-5b187bbdcf21 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d1554bc519b4fe4a2c28035459bc4b1)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:29.313096 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30a6816-1e69-4135-a2fb-5b187bbdcf21 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:29.361732 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-b30a6816-1e69-4135-a2fb-5b187bbdcf21 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d1554bc519b4fe4a2c28035459bc4b1', 'request_id': 'req-b30a6816-1e69-4135-a2fb-5b187bbdcf21'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:29.368272 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-b30a6816-1e69-4135-a2fb-5b187bbdcf21 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d1554bc519b4fe4a2c28035459bc4b1', 'request_id': 'req-b30a6816-1e69-4135-a2fb-5b187bbdcf21'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:29.374573 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-b30a6816-1e69-4135-a2fb-5b187bbdcf21 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d1554bc519b4fe4a2c28035459bc4b1', 'request_id': 'req-b30a6816-1e69-4135-a2fb-5b187bbdcf21'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:29.388369 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-b30a6816-1e69-4135-a2fb-5b187bbdcf21 admin admin] The token cache is being invalidate because project 3d1554bc519b4fe4a2c28035459bc4b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81591) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:14:29.388369 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-b30a6816-1e69-4135-a2fb-5b187bbdcf21 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:29.389052 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-b30a6816-1e69-4135-a2fb-5b187bbdcf21 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:29.390562 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1065/2110] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 9 12:14:29 2022] DELETE /identity/v3/projects/3d1554bc519b4fe4a2c28035459bc4b1 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:14:29.396601 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30696522-e80e-4330-9de8-ec19a43a1d15 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:29.397077 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30696522-e80e-4330-9de8-ec19a43a1d15 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:29.397220 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30696522-e80e-4330-9de8-ec19a43a1d15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:29.419311 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-30696522-e80e-4330-9de8-ec19a43a1d15 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:14:29.420626 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-30696522-e80e-4330-9de8-ec19a43a1d15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:29.528460 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1046/2111] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:14:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:14:29.536828 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fd1aeb-7f59-4a73-b7af-720cf6656473 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:29.537109 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fd1aeb-7f59-4a73-b7af-720cf6656473 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:29.537379 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fd1aeb-7f59-4a73-b7af-720cf6656473 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:29.562966 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-01fd1aeb-7f59-4a73-b7af-720cf6656473 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:14:29.564181 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-01fd1aeb-7f59-4a73-b7af-720cf6656473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:29.572610 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1066/2112] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:14:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:14:29.576969 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30696522-e80e-4330-9de8-ec19a43a1d15 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:29.588202 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30696522-e80e-4330-9de8-ec19a43a1d15 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:29.593111 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d9deba-2e0a-4917-8a0e-13c75a0a5605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:29.593761 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d9deba-2e0a-4917-8a0e-13c75a0a5605 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:29.594105 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d9deba-2e0a-4917-8a0e-13c75a0a5605 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:29.594445 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d9deba-2e0a-4917-8a0e-13c75a0a5605 admin admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:29.595190 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d9deba-2e0a-4917-8a0e-13c75a0a5605 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:14:29.595578 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d9deba-2e0a-4917-8a0e-13c75a0a5605 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:29.597273 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d9deba-2e0a-4917-8a0e-13c75a0a5605 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:29.613409 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1047/2113] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 9 12:14:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:14:29.614661 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fd1aeb-7f59-4a73-b7af-720cf6656473 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:29.626533 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fd1aeb-7f59-4a73-b7af-720cf6656473 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:29.632073 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baba6e51-538d-4eed-aded-7d7f3fb8ace3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:29.632733 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baba6e51-538d-4eed-aded-7d7f3fb8ace3 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:29.633035 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baba6e51-538d-4eed-aded-7d7f3fb8ace3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:29.633309 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baba6e51-538d-4eed-aded-7d7f3fb8ace3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:29.634016 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baba6e51-538d-4eed-aded-7d7f3fb8ace3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:29.636312 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baba6e51-538d-4eed-aded-7d7f3fb8ace3 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:29.671390 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1067/2114] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 9 12:14:29 2022] POST /identity/v3/projects => generated 409 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:14:29.675809 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d9deba-2e0a-4917-8a0e-13c75a0a5605 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:29.686395 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d9deba-2e0a-4917-8a0e-13c75a0a5605 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:29.745209 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbcead4-faf7-46c3-b5dd-1bd70174238e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:29.745808 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbcead4-faf7-46c3-b5dd-1bd70174238e admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:29.746087 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbcead4-faf7-46c3-b5dd-1bd70174238e admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:29.746353 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbcead4-faf7-46c3-b5dd-1bd70174238e admin admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:29.747017 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbcead4-faf7-46c3-b5dd-1bd70174238e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:29.748923 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbcead4-faf7-46c3-b5dd-1bd70174238e admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:29.769561 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1048/2115] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:14:29 2022] POST /identity/v3/users => generated 305 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:14:29.773666 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baba6e51-538d-4eed-aded-7d7f3fb8ace3 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:29.784154 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baba6e51-538d-4eed-aded-7d7f3fb8ace3 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:29.788754 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a9146b-85f3-409b-8205-c3ca4a145148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:29.789949 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a9146b-85f3-409b-8205-c3ca4a145148 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:29.790332 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a9146b-85f3-409b-8205-c3ca4a145148 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:29.790605 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a9146b-85f3-409b-8205-c3ca4a145148 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:29.791287 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a9146b-85f3-409b-8205-c3ca4a145148 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:29.792866 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a9146b-85f3-409b-8205-c3ca4a145148 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:29.802783 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1068/2116] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:14:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:14:29.807601 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbcead4-faf7-46c3-b5dd-1bd70174238e admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:29.817647 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbcead4-faf7-46c3-b5dd-1bd70174238e admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:29.822494 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1869f493-8adb-48c4-b5f1-ca22ccc178cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:29.823106 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1869f493-8adb-48c4-b5f1-ca22ccc178cd admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:29.823380 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1869f493-8adb-48c4-b5f1-ca22ccc178cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:29.823643 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1869f493-8adb-48c4-b5f1-ca22ccc178cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:29.824271 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1869f493-8adb-48c4-b5f1-ca22ccc178cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:29.825853 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1869f493-8adb-48c4-b5f1-ca22ccc178cd admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:29.838323 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1049/2117] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:14:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:14:29.842293 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a9146b-85f3-409b-8205-c3ca4a145148 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:29.853159 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a9146b-85f3-409b-8205-c3ca4a145148 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:29.857711 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc085c6-d712-4349-adaf-d97d2a01af05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:29.859060 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc085c6-d712-4349-adaf-d97d2a01af05 admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:29.859342 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc085c6-d712-4349-adaf-d97d2a01af05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:29.859611 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc085c6-d712-4349-adaf-d97d2a01af05 admin admin] PATH_INFO: `/v3/projects/84317ee8523e42c2b548d9740db75427/users/0b00b7a19c204480bdca90ba2ff8ce0f/roles/eecd2f160dd844f5906f3d345780c501` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:29.874753 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc085c6-d712-4349-adaf-d97d2a01af05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84317ee8523e42c2b548d9740db75427, user_id=0b00b7a19c204480bdca90ba2ff8ce0f, role_id=eecd2f160dd844f5906f3d345780c501)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:29.877416 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc085c6-d712-4349-adaf-d97d2a01af05 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:29.879572 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:14:29.892478 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1069/2118] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:14:29 2022] PUT /identity/v3/projects/84317ee8523e42c2b548d9740db75427/users/0b00b7a19c204480bdca90ba2ff8ce0f/roles/eecd2f160dd844f5906f3d345780c501 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:14:29.903326 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b7dc48-b5d5-4b7d-9ff9-7cf0c0e50c07 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:29.903608 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b7dc48-b5d5-4b7d-9ff9-7cf0c0e50c07 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:29.903878 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b7dc48-b5d5-4b7d-9ff9-7cf0c0e50c07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:29.931300 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-b3b7dc48-b5d5-4b7d-9ff9-7cf0c0e50c07 None None] MFA Rules not processed for user `0b00b7a19c204480bdca90ba2ff8ce0f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:14:29.932499 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-b3b7dc48-b5d5-4b7d-9ff9-7cf0c0e50c07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:29.974261 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:14:29.998613 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1050/2119] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:14:29 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:14:30.003298 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc085c6-d712-4349-adaf-d97d2a01af05 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:30.069352 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc085c6-d712-4349-adaf-d97d2a01af05 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:30.074094 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc146e6d-8f5a-4bd3-8947-76787d86ac1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:30.074686 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc146e6d-8f5a-4bd3-8947-76787d86ac1d admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:30.074989 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc146e6d-8f5a-4bd3-8947-76787d86ac1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:30.075258 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc146e6d-8f5a-4bd3-8947-76787d86ac1d admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:30.075887 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc146e6d-8f5a-4bd3-8947-76787d86ac1d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:30.077809 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc146e6d-8f5a-4bd3-8947-76787d86ac1d admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:30.105136 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1070/2120] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 9 12:14:30 2022] POST /identity/v3/projects => generated 407 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:14:30.109189 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b7dc48-b5d5-4b7d-9ff9-7cf0c0e50c07 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:30.119965 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b7dc48-b5d5-4b7d-9ff9-7cf0c0e50c07 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:30.176266 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d32ae67-1986-4da6-b679-aaf50f7484e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:30.176875 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d32ae67-1986-4da6-b679-aaf50f7484e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:30.177156 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d32ae67-1986-4da6-b679-aaf50f7484e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:30.177429 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d32ae67-1986-4da6-b679-aaf50f7484e5 admin admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:30.178055 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d32ae67-1986-4da6-b679-aaf50f7484e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:30.180061 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d32ae67-1986-4da6-b679-aaf50f7484e5 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:30.201622 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1051/2121] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:14:30 2022] POST /identity/v3/users => generated 310 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:14:30.205578 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc146e6d-8f5a-4bd3-8947-76787d86ac1d admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:30.216289 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc146e6d-8f5a-4bd3-8947-76787d86ac1d admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:30.220866 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859a2394-f57b-4cf1-8441-1d33ad9e602c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:30.221460 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859a2394-f57b-4cf1-8441-1d33ad9e602c admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:30.221742 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859a2394-f57b-4cf1-8441-1d33ad9e602c admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:30.222010 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859a2394-f57b-4cf1-8441-1d33ad9e602c admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:30.222654 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859a2394-f57b-4cf1-8441-1d33ad9e602c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:30.224280 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859a2394-f57b-4cf1-8441-1d33ad9e602c admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:30.235636 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1071/2122] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:14:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:14:30.239607 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d32ae67-1986-4da6-b679-aaf50f7484e5 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:30.249821 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d32ae67-1986-4da6-b679-aaf50f7484e5 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:30.254518 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c277b2f-4f07-43f6-a863-b252b6f9de83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:30.255225 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c277b2f-4f07-43f6-a863-b252b6f9de83 admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:30.255501 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c277b2f-4f07-43f6-a863-b252b6f9de83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:30.255768 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c277b2f-4f07-43f6-a863-b252b6f9de83 admin admin] PATH_INFO: `/v3/projects/460c2309fefe4d1ead7c90e105bfb27f/users/b386fcb2f43c461b99426ddb755182c4/roles/3f2ca33902a04b6dbc2ef7d06d7289d3` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:30.270962 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c277b2f-4f07-43f6-a863-b252b6f9de83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=460c2309fefe4d1ead7c90e105bfb27f, user_id=b386fcb2f43c461b99426ddb755182c4, role_id=3f2ca33902a04b6dbc2ef7d06d7289d3)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:30.274200 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c277b2f-4f07-43f6-a863-b252b6f9de83 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:30.287896 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1052/2123] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:14:30 2022] PUT /identity/v3/projects/460c2309fefe4d1ead7c90e105bfb27f/users/b386fcb2f43c461b99426ddb755182c4/roles/3f2ca33902a04b6dbc2ef7d06d7289d3 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:14:30.292686 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859a2394-f57b-4cf1-8441-1d33ad9e602c admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:30.302434 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859a2394-f57b-4cf1-8441-1d33ad9e602c admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:30.358018 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88004326-c10e-45fb-b1d3-031400016e97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:30.358546 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88004326-c10e-45fb-b1d3-031400016e97 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:30.358826 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88004326-c10e-45fb-b1d3-031400016e97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:30.359135 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88004326-c10e-45fb-b1d3-031400016e97 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:30.359772 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88004326-c10e-45fb-b1d3-031400016e97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:30.361367 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88004326-c10e-45fb-b1d3-031400016e97 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:30.371098 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1072/2124] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:14:30 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:14:30.375325 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c277b2f-4f07-43f6-a863-b252b6f9de83 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:30.385975 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c277b2f-4f07-43f6-a863-b252b6f9de83 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:30.390596 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498478cf-ce81-4eb0-83c5-fbedf66ca5b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:30.391278 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498478cf-ce81-4eb0-83c5-fbedf66ca5b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:30.391561 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498478cf-ce81-4eb0-83c5-fbedf66ca5b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:30.391832 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498478cf-ce81-4eb0-83c5-fbedf66ca5b1 admin admin] PATH_INFO: `/v3/projects/460c2309fefe4d1ead7c90e105bfb27f/users/b386fcb2f43c461b99426ddb755182c4/roles/3f2ca33902a04b6dbc2ef7d06d7289d3` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:30.393712 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498478cf-ce81-4eb0-83c5-fbedf66ca5b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=460c2309fefe4d1ead7c90e105bfb27f, user_id=b386fcb2f43c461b99426ddb755182c4, role_id=3f2ca33902a04b6dbc2ef7d06d7289d3)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:30.396404 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498478cf-ce81-4eb0-83c5-fbedf66ca5b1 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:30.407116 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1053/2125] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:14:30 2022] PUT /identity/v3/projects/460c2309fefe4d1ead7c90e105bfb27f/users/b386fcb2f43c461b99426ddb755182c4/roles/3f2ca33902a04b6dbc2ef7d06d7289d3 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:14:30.418597 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5938aa-7321-43d7-a079-b91e33f2bbdc None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:30.418886 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5938aa-7321-43d7-a079-b91e33f2bbdc None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:30.419198 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5938aa-7321-43d7-a079-b91e33f2bbdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:30.447373 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-dc5938aa-7321-43d7-a079-b91e33f2bbdc None None] MFA Rules not processed for user `b386fcb2f43c461b99426ddb755182c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:14:30.448685 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-dc5938aa-7321-43d7-a079-b91e33f2bbdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:30.497422 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:14:30.517735 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1073/2126] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:14:30 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:14:30.529416 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498478cf-ce81-4eb0-83c5-fbedf66ca5b1 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:30.531167 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-498478cf-ce81-4eb0-83c5-fbedf66ca5b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:30.591381 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498478cf-ce81-4eb0-83c5-fbedf66ca5b1 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:30.595018 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a62b2d-d104-4623-bd30-dddb8d4d3274 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535c258100df40b58a5ee44b2c21d621', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:30.595621 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a62b2d-d104-4623-bd30-dddb8d4d3274 service cinder] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:30.595901 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a62b2d-d104-4623-bd30-dddb8d4d3274 service cinder] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:30.596173 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a62b2d-d104-4623-bd30-dddb8d4d3274 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:30.605251 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a62b2d-d104-4623-bd30-dddb8d4d3274 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:30.606674 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a62b2d-d104-4623-bd30-dddb8d4d3274 service cinder] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:30.616105 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1054/2127] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:14:30 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:14:30.782802 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5938aa-7321-43d7-a079-b91e33f2bbdc None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:30.792142 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5938aa-7321-43d7-a079-b91e33f2bbdc None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:30.795659 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea49ecf-3511-460c-860f-eecb9a7e6292 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535c258100df40b58a5ee44b2c21d621', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:30.796258 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea49ecf-3511-460c-860f-eecb9a7e6292 service cinder] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:30.796534 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea49ecf-3511-460c-860f-eecb9a7e6292 service cinder] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:30.796808 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea49ecf-3511-460c-860f-eecb9a7e6292 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:30.855284 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea49ecf-3511-460c-860f-eecb9a7e6292 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:30.856711 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea49ecf-3511-460c-860f-eecb9a7e6292 service cinder] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:30.865665 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1074/2128] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:14:30 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:14:33.015673 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a62b2d-d104-4623-bd30-dddb8d4d3274 service cinder] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:33.017113 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-d3a62b2d-d104-4623-bd30-dddb8d4d3274 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:33.028623 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a62b2d-d104-4623-bd30-dddb8d4d3274 service cinder] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:33.032229 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4d1443-2eb0-4899-b7ce-3611ec1cf98f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535c258100df40b58a5ee44b2c21d621', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:33.032819 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4d1443-2eb0-4899-b7ce-3611ec1cf98f service cinder] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:33.033096 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4d1443-2eb0-4899-b7ce-3611ec1cf98f service cinder] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:33.033366 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4d1443-2eb0-4899-b7ce-3611ec1cf98f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:33.035058 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-ae4d1443-2eb0-4899-b7ce-3611ec1cf98f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:33.068995 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:14:33.098391 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4d1443-2eb0-4899-b7ce-3611ec1cf98f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:33.099866 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4d1443-2eb0-4899-b7ce-3611ec1cf98f service cinder] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:33.110074 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1055/2129] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:14:33 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:14:34.050531 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73bbd30-4d4e-4cd8-a00e-11c1c92fb1fd None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:34.051035 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73bbd30-4d4e-4cd8-a00e-11c1c92fb1fd None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:34.051278 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73bbd30-4d4e-4cd8-a00e-11c1c92fb1fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:34.071446 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-c73bbd30-4d4e-4cd8-a00e-11c1c92fb1fd None None] MFA Rules not processed for user `913344053b6244c584140807ab2149cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:14:34.072654 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-c73bbd30-4d4e-4cd8-a00e-11c1c92fb1fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:34.126123 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1075/2130] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:14:34 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:14:40.151693 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4d1443-2eb0-4899-b7ce-3611ec1cf98f service cinder] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:40.153344 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-ae4d1443-2eb0-4899-b7ce-3611ec1cf98f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:40.270749 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4d1443-2eb0-4899-b7ce-3611ec1cf98f service cinder] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:40.279068 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b9b447-432b-4aff-a8cb-872cc184ec9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:40.279890 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b9b447-432b-4aff-a8cb-872cc184ec9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:40.280274 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b9b447-432b-4aff-a8cb-872cc184ec9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:40.280643 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b9b447-432b-4aff-a8cb-872cc184ec9e admin admin] PATH_INFO: `/v3/users/b102681c7ca44cefbe8b1bdd34bbd0fb` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:40.282864 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b9b447-432b-4aff-a8cb-872cc184ec9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b102681c7ca44cefbe8b1bdd34bbd0fb)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:40.285175 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b9b447-432b-4aff-a8cb-872cc184ec9e admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:40.371815 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-75b9b447-432b-4aff-a8cb-872cc184ec9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b102681c7ca44cefbe8b1bdd34bbd0fb'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:40.386224 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-75b9b447-432b-4aff-a8cb-872cc184ec9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b102681c7ca44cefbe8b1bdd34bbd0fb'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:40.394198 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-75b9b447-432b-4aff-a8cb-872cc184ec9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b102681c7ca44cefbe8b1bdd34bbd0fb'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:40.405267 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-75b9b447-432b-4aff-a8cb-872cc184ec9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b102681c7ca44cefbe8b1bdd34bbd0fb'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:40.406103 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-75b9b447-432b-4aff-a8cb-872cc184ec9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b102681c7ca44cefbe8b1bdd34bbd0fb'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:40.416377 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1056/2131] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 9 12:14:40 2022] DELETE /identity/v3/users/b102681c7ca44cefbe8b1bdd34bbd0fb => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:14:40.418285 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932758d9-2852-4622-8d21-345d2b9656f2 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:40.418732 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932758d9-2852-4622-8d21-345d2b9656f2 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:40.419077 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932758d9-2852-4622-8d21-345d2b9656f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:40.441302 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-932758d9-2852-4622-8d21-345d2b9656f2 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:14:40.442538 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-932758d9-2852-4622-8d21-345d2b9656f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:40.513210 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1076/2132] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:14:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:14:40.522134 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b9b447-432b-4aff-a8cb-872cc184ec9e admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:40.525219 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-75b9b447-432b-4aff-a8cb-872cc184ec9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:40.599034 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b9b447-432b-4aff-a8cb-872cc184ec9e admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:40.603369 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fdb1b9-3fd5-4117-8fd4-ba710f50e323 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:40.603988 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fdb1b9-3fd5-4117-8fd4-ba710f50e323 service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:40.604271 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fdb1b9-3fd5-4117-8fd4-ba710f50e323 service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:40.604542 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fdb1b9-3fd5-4117-8fd4-ba710f50e323 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:40.610720 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:14:40.614867 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fdb1b9-3fd5-4117-8fd4-ba710f50e323 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:40.616872 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fdb1b9-3fd5-4117-8fd4-ba710f50e323 service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:40.629156 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1057/2133] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:14:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:14:40.940046 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932758d9-2852-4622-8d21-345d2b9656f2 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:40.942728 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-932758d9-2852-4622-8d21-345d2b9656f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:40.957566 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932758d9-2852-4622-8d21-345d2b9656f2 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:40.964066 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce6852f-f16d-4307-bc23-8a8bf04100cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:40.964703 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce6852f-f16d-4307-bc23-8a8bf04100cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:40.964985 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce6852f-f16d-4307-bc23-8a8bf04100cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:40.965350 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce6852f-f16d-4307-bc23-8a8bf04100cb admin admin] PATH_INFO: `/v3/projects/b4eabdd7ac6b4e6983c44fb0f6f031b1` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:40.966496 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce6852f-f16d-4307-bc23-8a8bf04100cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4eabdd7ac6b4e6983c44fb0f6f031b1)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:40.968730 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce6852f-f16d-4307-bc23-8a8bf04100cb admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:41.006547 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:14:41.025261 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-1ce6852f-f16d-4307-bc23-8a8bf04100cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4eabdd7ac6b4e6983c44fb0f6f031b1', 'request_id': 'req-1ce6852f-f16d-4307-bc23-8a8bf04100cb'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:41.031188 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-1ce6852f-f16d-4307-bc23-8a8bf04100cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4eabdd7ac6b4e6983c44fb0f6f031b1', 'request_id': 'req-1ce6852f-f16d-4307-bc23-8a8bf04100cb'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:41.039251 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-1ce6852f-f16d-4307-bc23-8a8bf04100cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4eabdd7ac6b4e6983c44fb0f6f031b1', 'request_id': 'req-1ce6852f-f16d-4307-bc23-8a8bf04100cb'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:41.052614 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-1ce6852f-f16d-4307-bc23-8a8bf04100cb admin admin] The token cache is being invalidate because project b4eabdd7ac6b4e6983c44fb0f6f031b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81591) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:14:41.052900 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-1ce6852f-f16d-4307-bc23-8a8bf04100cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:41.053526 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-1ce6852f-f16d-4307-bc23-8a8bf04100cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:41.055607 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1077/2134] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 9 12:14:40 2022] DELETE /identity/v3/projects/b4eabdd7ac6b4e6983c44fb0f6f031b1 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:14:41.061083 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ca2a9a-64d1-455d-816d-e2f9a5936446 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:41.061374 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ca2a9a-64d1-455d-816d-e2f9a5936446 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:41.061650 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ca2a9a-64d1-455d-816d-e2f9a5936446 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:41.081970 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-55ca2a9a-64d1-455d-816d-e2f9a5936446 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:14:41.083203 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-55ca2a9a-64d1-455d-816d-e2f9a5936446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:41.142762 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1058/2135] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:14:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:14:41.151599 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef040a5-961d-427d-b242-53ed72a8d0d7 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:41.151883 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef040a5-961d-427d-b242-53ed72a8d0d7 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:41.152156 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef040a5-961d-427d-b242-53ed72a8d0d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:41.172622 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-4ef040a5-961d-427d-b242-53ed72a8d0d7 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:14:41.173824 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-4ef040a5-961d-427d-b242-53ed72a8d0d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:41.182262 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1078/2136] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:14:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:14:41.186632 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ca2a9a-64d1-455d-816d-e2f9a5936446 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:41.198010 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ca2a9a-64d1-455d-816d-e2f9a5936446 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:41.202646 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a54fd9-c49b-48a9-943a-e0ae46182d39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:41.203355 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a54fd9-c49b-48a9-943a-e0ae46182d39 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:41.203535 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a54fd9-c49b-48a9-943a-e0ae46182d39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:41.203804 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a54fd9-c49b-48a9-943a-e0ae46182d39 admin admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:41.204442 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a54fd9-c49b-48a9-943a-e0ae46182d39 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:14:41.204766 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a54fd9-c49b-48a9-943a-e0ae46182d39 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:41.206314 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a54fd9-c49b-48a9-943a-e0ae46182d39 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:41.218763 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1059/2137] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 9 12:14:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:14:41.222989 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef040a5-961d-427d-b242-53ed72a8d0d7 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:41.232699 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef040a5-961d-427d-b242-53ed72a8d0d7 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:41.237320 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b03074-f5f7-45fd-aa6f-8bf20ba77a48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:41.237916 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b03074-f5f7-45fd-aa6f-8bf20ba77a48 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:41.238193 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b03074-f5f7-45fd-aa6f-8bf20ba77a48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:41.238465 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b03074-f5f7-45fd-aa6f-8bf20ba77a48 admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:41.239113 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b03074-f5f7-45fd-aa6f-8bf20ba77a48 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:41.241015 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b03074-f5f7-45fd-aa6f-8bf20ba77a48 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:41.267544 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1079/2138] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 9 12:14:41 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:14:41.271586 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a54fd9-c49b-48a9-943a-e0ae46182d39 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:41.282080 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a54fd9-c49b-48a9-943a-e0ae46182d39 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:41.350225 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576a8ced-90a9-45d6-88d0-6afe7c7e1025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:41.351931 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576a8ced-90a9-45d6-88d0-6afe7c7e1025 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:41.352196 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576a8ced-90a9-45d6-88d0-6afe7c7e1025 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:41.352491 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576a8ced-90a9-45d6-88d0-6afe7c7e1025 admin admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:41.353122 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576a8ced-90a9-45d6-88d0-6afe7c7e1025 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:41.355093 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576a8ced-90a9-45d6-88d0-6afe7c7e1025 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:41.382492 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1060/2139] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:14:41 2022] POST /identity/v3/users => generated 287 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:14:41.383951 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b03074-f5f7-45fd-aa6f-8bf20ba77a48 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:41.394380 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b03074-f5f7-45fd-aa6f-8bf20ba77a48 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:41.399546 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09d07a0-a5d9-4331-baa9-14c6ec86ed94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:41.400147 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09d07a0-a5d9-4331-baa9-14c6ec86ed94 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:41.400427 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09d07a0-a5d9-4331-baa9-14c6ec86ed94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:41.400699 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09d07a0-a5d9-4331-baa9-14c6ec86ed94 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:41.401367 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09d07a0-a5d9-4331-baa9-14c6ec86ed94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:41.403002 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09d07a0-a5d9-4331-baa9-14c6ec86ed94 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:41.416870 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1080/2140] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:14:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 12:14:41.418166 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576a8ced-90a9-45d6-88d0-6afe7c7e1025 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:41.427974 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576a8ced-90a9-45d6-88d0-6afe7c7e1025 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:41.433014 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e753a1-3823-456d-8cb7-b43c32c63b6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:41.433640 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e753a1-3823-456d-8cb7-b43c32c63b6b admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:41.433917 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e753a1-3823-456d-8cb7-b43c32c63b6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:41.434443 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e753a1-3823-456d-8cb7-b43c32c63b6b admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:41.435153 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e753a1-3823-456d-8cb7-b43c32c63b6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:41.436798 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e753a1-3823-456d-8cb7-b43c32c63b6b admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:41.448710 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1061/2141] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:14:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:14:41.453177 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09d07a0-a5d9-4331-baa9-14c6ec86ed94 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:41.463894 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09d07a0-a5d9-4331-baa9-14c6ec86ed94 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:41.468607 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07973796-563e-4db1-9eeb-1dec005e5b09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:41.469269 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07973796-563e-4db1-9eeb-1dec005e5b09 admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:41.469550 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07973796-563e-4db1-9eeb-1dec005e5b09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:41.469854 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07973796-563e-4db1-9eeb-1dec005e5b09 admin admin] PATH_INFO: `/v3/projects/6b0570ae08fc40ccb3dd942591098d34/users/9750f671d674447f9fbf36d382573c37/roles/eecd2f160dd844f5906f3d345780c501` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:41.484741 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07973796-563e-4db1-9eeb-1dec005e5b09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b0570ae08fc40ccb3dd942591098d34, user_id=9750f671d674447f9fbf36d382573c37, role_id=eecd2f160dd844f5906f3d345780c501)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:41.487448 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07973796-563e-4db1-9eeb-1dec005e5b09 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:41.500981 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1081/2142] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:14:41 2022] PUT /identity/v3/projects/6b0570ae08fc40ccb3dd942591098d34/users/9750f671d674447f9fbf36d382573c37/roles/eecd2f160dd844f5906f3d345780c501 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:14:41.505956 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9362f12b-7a97-4335-baad-efc545bc14cc None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:41.506239 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9362f12b-7a97-4335-baad-efc545bc14cc None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:41.506533 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9362f12b-7a97-4335-baad-efc545bc14cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:41.526622 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-9362f12b-7a97-4335-baad-efc545bc14cc None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:14:41.528300 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-9362f12b-7a97-4335-baad-efc545bc14cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:41.578724 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:14:41.589428 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1062/2143] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:14:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:14:41.599050 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07973796-563e-4db1-9eeb-1dec005e5b09 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:41.600759 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-07973796-563e-4db1-9eeb-1dec005e5b09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:41.657398 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07973796-563e-4db1-9eeb-1dec005e5b09 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:41.660998 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ad1486-e76c-4d78-9538-927db0709cbc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:41.661593 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ad1486-e76c-4d78-9538-927db0709cbc service neutron] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:41.661867 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ad1486-e76c-4d78-9538-927db0709cbc service neutron] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:41.662136 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ad1486-e76c-4d78-9538-927db0709cbc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:41.670884 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ad1486-e76c-4d78-9538-927db0709cbc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:41.672323 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ad1486-e76c-4d78-9538-927db0709cbc service neutron] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:41.681825 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1082/2144] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:14:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:14:45.920999 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9362f12b-7a97-4335-baad-efc545bc14cc None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:45.923160 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-9362f12b-7a97-4335-baad-efc545bc14cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:45.943387 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9362f12b-7a97-4335-baad-efc545bc14cc None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:45.960552 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ec104d-09ba-4060-a0a3-964d30106a93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:45.961482 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ec104d-09ba-4060-a0a3-964d30106a93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:45.961890 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ec104d-09ba-4060-a0a3-964d30106a93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:45.962301 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ec104d-09ba-4060-a0a3-964d30106a93 admin admin] PATH_INFO: `/v3/users/0b00b7a19c204480bdca90ba2ff8ce0f` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:45.963744 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ec104d-09ba-4060-a0a3-964d30106a93 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b00b7a19c204480bdca90ba2ff8ce0f)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:45.966295 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ec104d-09ba-4060-a0a3-964d30106a93 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:46.042448 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-c2ec104d-09ba-4060-a0a3-964d30106a93 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b00b7a19c204480bdca90ba2ff8ce0f'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:46.079650 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-c2ec104d-09ba-4060-a0a3-964d30106a93 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b00b7a19c204480bdca90ba2ff8ce0f'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:46.100193 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-c2ec104d-09ba-4060-a0a3-964d30106a93 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b00b7a19c204480bdca90ba2ff8ce0f'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:46.114287 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-c2ec104d-09ba-4060-a0a3-964d30106a93 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b00b7a19c204480bdca90ba2ff8ce0f'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:46.115107 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-c2ec104d-09ba-4060-a0a3-964d30106a93 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b00b7a19c204480bdca90ba2ff8ce0f'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:46.130715 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1063/2145] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 9 12:14:45 2022] DELETE /identity/v3/users/0b00b7a19c204480bdca90ba2ff8ce0f => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:14:46.136438 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa202d8-eee4-4ab9-b77c-ac05b1383969 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:46.136929 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa202d8-eee4-4ab9-b77c-ac05b1383969 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:46.137021 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa202d8-eee4-4ab9-b77c-ac05b1383969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:46.164791 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-caa202d8-eee4-4ab9-b77c-ac05b1383969 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:14:46.166058 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-caa202d8-eee4-4ab9-b77c-ac05b1383969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:46.245192 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:14:46.295591 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1083/2146] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:14:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:14:46.296971 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ec104d-09ba-4060-a0a3-964d30106a93 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:46.298669 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-c2ec104d-09ba-4060-a0a3-964d30106a93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:46.403762 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ec104d-09ba-4060-a0a3-964d30106a93 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:46.408187 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a79100-7b50-4f3a-a4d2-78ace635fbe8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:46.409184 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a79100-7b50-4f3a-a4d2-78ace635fbe8 service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:46.409465 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a79100-7b50-4f3a-a4d2-78ace635fbe8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:46.409738 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a79100-7b50-4f3a-a4d2-78ace635fbe8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:46.421924 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a79100-7b50-4f3a-a4d2-78ace635fbe8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:46.423437 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a79100-7b50-4f3a-a4d2-78ace635fbe8 service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:46.447465 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1064/2147] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:14:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:14:46.869158 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa202d8-eee4-4ab9-b77c-ac05b1383969 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:46.870814 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-caa202d8-eee4-4ab9-b77c-ac05b1383969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:46.888940 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa202d8-eee4-4ab9-b77c-ac05b1383969 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:46.894355 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d9c9bd-ff71-4cbc-9835-5de7e040de0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:46.895049 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d9c9bd-ff71-4cbc-9835-5de7e040de0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:46.895333 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d9c9bd-ff71-4cbc-9835-5de7e040de0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:46.895607 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d9c9bd-ff71-4cbc-9835-5de7e040de0c admin admin] PATH_INFO: `/v3/projects/84317ee8523e42c2b548d9740db75427` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:46.897093 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d9c9bd-ff71-4cbc-9835-5de7e040de0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=84317ee8523e42c2b548d9740db75427)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:46.900083 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d9c9bd-ff71-4cbc-9835-5de7e040de0c admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:47.000445 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-36d9c9bd-ff71-4cbc-9835-5de7e040de0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84317ee8523e42c2b548d9740db75427', 'request_id': 'req-36d9c9bd-ff71-4cbc-9835-5de7e040de0c'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:47.008998 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-36d9c9bd-ff71-4cbc-9835-5de7e040de0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84317ee8523e42c2b548d9740db75427', 'request_id': 'req-36d9c9bd-ff71-4cbc-9835-5de7e040de0c'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:47.017825 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-36d9c9bd-ff71-4cbc-9835-5de7e040de0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84317ee8523e42c2b548d9740db75427', 'request_id': 'req-36d9c9bd-ff71-4cbc-9835-5de7e040de0c'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:47.039923 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-36d9c9bd-ff71-4cbc-9835-5de7e040de0c admin admin] The token cache is being invalidate because project 84317ee8523e42c2b548d9740db75427 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81591) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:14:47.040223 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-36d9c9bd-ff71-4cbc-9835-5de7e040de0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:47.040927 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-36d9c9bd-ff71-4cbc-9835-5de7e040de0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:47.045811 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1084/2148] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 9 12:14:46 2022] DELETE /identity/v3/projects/84317ee8523e42c2b548d9740db75427 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:14:47.046876 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a79100-7b50-4f3a-a4d2-78ace635fbe8 service neutron] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:47.058433 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a79100-7b50-4f3a-a4d2-78ace635fbe8 service neutron] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:47.155180 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45f62c0-eb9c-461f-b54d-6b4be6b1b88f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:47.156006 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45f62c0-eb9c-461f-b54d-6b4be6b1b88f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:47.156127 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45f62c0-eb9c-461f-b54d-6b4be6b1b88f admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:47.156387 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45f62c0-eb9c-461f-b54d-6b4be6b1b88f admin admin] PATH_INFO: `/v3/users/b386fcb2f43c461b99426ddb755182c4` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:47.157733 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45f62c0-eb9c-461f-b54d-6b4be6b1b88f admin admin] RBAC: Authorizing `identity:delete_user(user_id=b386fcb2f43c461b99426ddb755182c4)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:47.159881 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45f62c0-eb9c-461f-b54d-6b4be6b1b88f admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:47.186855 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:14:47.263993 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-f45f62c0-eb9c-461f-b54d-6b4be6b1b88f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b386fcb2f43c461b99426ddb755182c4'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:47.277618 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-f45f62c0-eb9c-461f-b54d-6b4be6b1b88f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b386fcb2f43c461b99426ddb755182c4'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:47.285850 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-f45f62c0-eb9c-461f-b54d-6b4be6b1b88f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b386fcb2f43c461b99426ddb755182c4'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:47.301471 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-f45f62c0-eb9c-461f-b54d-6b4be6b1b88f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b386fcb2f43c461b99426ddb755182c4'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:47.302181 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-f45f62c0-eb9c-461f-b54d-6b4be6b1b88f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b386fcb2f43c461b99426ddb755182c4'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:47.320649 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1065/2149] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 9 12:14:47 2022] DELETE /identity/v3/users/b386fcb2f43c461b99426ddb755182c4 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:14:47.399363 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26007844-6122-496f-bf58-64fb406131b0 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:47.399859 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26007844-6122-496f-bf58-64fb406131b0 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:47.399954 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26007844-6122-496f-bf58-64fb406131b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:47.431819 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-26007844-6122-496f-bf58-64fb406131b0 None None] MFA Rules not processed for user `9750f671d674447f9fbf36d382573c37`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:14:47.433061 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-26007844-6122-496f-bf58-64fb406131b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:47.517175 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1085/2150] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:14:47 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:14:47.521767 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45f62c0-eb9c-461f-b54d-6b4be6b1b88f admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:47.524126 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-f45f62c0-eb9c-461f-b54d-6b4be6b1b88f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:47.596920 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45f62c0-eb9c-461f-b54d-6b4be6b1b88f admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:47.601619 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc64865-094b-4838-8dc6-6361488b8dc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:47.602234 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc64865-094b-4838-8dc6-6361488b8dc9 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:47.602512 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc64865-094b-4838-8dc6-6361488b8dc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:47.602778 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc64865-094b-4838-8dc6-6361488b8dc9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:47.603500 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc64865-094b-4838-8dc6-6361488b8dc9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:47.605428 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc64865-094b-4838-8dc6-6361488b8dc9 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:47.649235 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26007844-6122-496f-bf58-64fb406131b0 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:47.651653 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1066/2151] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 9 12:14:47 2022] POST /identity/v3/projects => generated 371 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:14:47.654707 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-26007844-6122-496f-bf58-64fb406131b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:47.655172 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc64865-094b-4838-8dc6-6361488b8dc9 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:47.680809 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26007844-6122-496f-bf58-64fb406131b0 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:47.758827 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1285a2cd-623d-400d-b478-75e573f50ae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:47.759507 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1285a2cd-623d-400d-b478-75e573f50ae3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:47.759839 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1285a2cd-623d-400d-b478-75e573f50ae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:47.760044 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1285a2cd-623d-400d-b478-75e573f50ae3 admin admin] PATH_INFO: `/v3/projects/460c2309fefe4d1ead7c90e105bfb27f` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:47.761234 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1285a2cd-623d-400d-b478-75e573f50ae3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=460c2309fefe4d1ead7c90e105bfb27f)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:47.763260 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1285a2cd-623d-400d-b478-75e573f50ae3 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:47.770662 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc64865-094b-4838-8dc6-6361488b8dc9 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:47.777200 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9996ee-f78d-4eff-9c65-4abba9a141ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:47.779063 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9996ee-f78d-4eff-9c65-4abba9a141ae admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:47.779540 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9996ee-f78d-4eff-9c65-4abba9a141ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:47.779796 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:14:47.779918 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9996ee-f78d-4eff-9c65-4abba9a141ae admin admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:47.781093 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9996ee-f78d-4eff-9c65-4abba9a141ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:47.783317 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9996ee-f78d-4eff-9c65-4abba9a141ae admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:47.836609 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1067/2152] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:14:47 2022] POST /identity/v3/users => generated 292 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:14:47.840339 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9996ee-f78d-4eff-9c65-4abba9a141ae admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:47.843742 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-1285a2cd-623d-400d-b478-75e573f50ae3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '460c2309fefe4d1ead7c90e105bfb27f', 'request_id': 'req-1285a2cd-623d-400d-b478-75e573f50ae3'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:47.850291 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9996ee-f78d-4eff-9c65-4abba9a141ae admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:47.851440 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-1285a2cd-623d-400d-b478-75e573f50ae3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '460c2309fefe4d1ead7c90e105bfb27f', 'request_id': 'req-1285a2cd-623d-400d-b478-75e573f50ae3'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:47.856099 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:DELETE=9,keystone:SELECT=86,keystone:INSERT=5 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:14:47.860047 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-1285a2cd-623d-400d-b478-75e573f50ae3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '460c2309fefe4d1ead7c90e105bfb27f', 'request_id': 'req-1285a2cd-623d-400d-b478-75e573f50ae3'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:47.878012 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-1285a2cd-623d-400d-b478-75e573f50ae3 admin admin] The token cache is being invalidate because project 460c2309fefe4d1ead7c90e105bfb27f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81591) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:14:47.878314 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-1285a2cd-623d-400d-b478-75e573f50ae3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:47.878989 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-1285a2cd-623d-400d-b478-75e573f50ae3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:47.880512 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1086/2153] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 9 12:14:47 2022] DELETE /identity/v3/projects/460c2309fefe4d1ead7c90e105bfb27f => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:14:47.886322 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b934f60-5779-4635-9a90-d58ee4d31404 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:47.886603 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b934f60-5779-4635-9a90-d58ee4d31404 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:47.886874 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b934f60-5779-4635-9a90-d58ee4d31404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:47.909706 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-5b934f60-5779-4635-9a90-d58ee4d31404 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:14:47.911050 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-5b934f60-5779-4635-9a90-d58ee4d31404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:47.920136 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83d5fca-df9e-4b95-b648-327114adc0a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:47.920754 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83d5fca-df9e-4b95-b648-327114adc0a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:47.921035 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83d5fca-df9e-4b95-b648-327114adc0a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:47.921307 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83d5fca-df9e-4b95-b648-327114adc0a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:47.921955 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83d5fca-df9e-4b95-b648-327114adc0a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:47.923643 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83d5fca-df9e-4b95-b648-327114adc0a2 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:47.938225 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1068/2154] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:14:47 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 12:14:47.942268 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83d5fca-df9e-4b95-b648-327114adc0a2 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:47.944000 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-f83d5fca-df9e-4b95-b648-327114adc0a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:47.958229 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83d5fca-df9e-4b95-b648-327114adc0a2 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:47.964753 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec8b299-cddd-4890-b94e-7eec31354873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:47.965441 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec8b299-cddd-4890-b94e-7eec31354873 admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:47.965738 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec8b299-cddd-4890-b94e-7eec31354873 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:47.966181 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec8b299-cddd-4890-b94e-7eec31354873 admin admin] PATH_INFO: `/v3/projects/2111959fe7e94015bd8f0383ab5c6f46/users/9039b37485b44800b2aff38093793ccb/roles/3f2ca33902a04b6dbc2ef7d06d7289d3` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:47.983595 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1087/2155] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:14:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:14:47.987926 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec8b299-cddd-4890-b94e-7eec31354873 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2111959fe7e94015bd8f0383ab5c6f46, user_id=9039b37485b44800b2aff38093793ccb, role_id=3f2ca33902a04b6dbc2ef7d06d7289d3)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:47.990884 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec8b299-cddd-4890-b94e-7eec31354873 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:47.993496 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf24ff8c-1add-4a08-bc0f-7d8c8859eece None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:47.993788 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf24ff8c-1add-4a08-bc0f-7d8c8859eece None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:47.994057 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf24ff8c-1add-4a08-bc0f-7d8c8859eece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:48.006670 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1069/2156] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:14:47 2022] PUT /identity/v3/projects/2111959fe7e94015bd8f0383ab5c6f46/users/9039b37485b44800b2aff38093793ccb/roles/3f2ca33902a04b6dbc2ef7d06d7289d3 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:14:48.010892 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec8b299-cddd-4890-b94e-7eec31354873 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:48.015938 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-bf24ff8c-1add-4a08-bc0f-7d8c8859eece None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:14:48.017327 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-bf24ff8c-1add-4a08-bc0f-7d8c8859eece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:48.074973 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec8b299-cddd-4890-b94e-7eec31354873 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:48.077120 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1088/2157] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:14:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:14:48.081399 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf24ff8c-1add-4a08-bc0f-7d8c8859eece None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:48.081587 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c19611d-b06a-47e6-9938-ff4ec982a58d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:48.082196 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c19611d-b06a-47e6-9938-ff4ec982a58d admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:48.082472 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c19611d-b06a-47e6-9938-ff4ec982a58d admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:48.082741 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c19611d-b06a-47e6-9938-ff4ec982a58d admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:48.083411 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c19611d-b06a-47e6-9938-ff4ec982a58d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:48.085605 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c19611d-b06a-47e6-9938-ff4ec982a58d admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:48.091296 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf24ff8c-1add-4a08-bc0f-7d8c8859eece None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:48.095828 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1070/2158] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:14:48 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 12:14:48.096225 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f0b1cf-cb69-4b21-932e-2c8dc5707c61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:48.096822 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f0b1cf-cb69-4b21-932e-2c8dc5707c61 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:48.097097 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f0b1cf-cb69-4b21-932e-2c8dc5707c61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:48.097364 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f0b1cf-cb69-4b21-932e-2c8dc5707c61 admin admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:48.097999 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f0b1cf-cb69-4b21-932e-2c8dc5707c61 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:14:48.098299 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f0b1cf-cb69-4b21-932e-2c8dc5707c61 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:48.099760 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c19611d-b06a-47e6-9938-ff4ec982a58d admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:48.099955 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f0b1cf-cb69-4b21-932e-2c8dc5707c61 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:48.111720 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c19611d-b06a-47e6-9938-ff4ec982a58d admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:48.112459 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1089/2159] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 9 12:14:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:14:48.117164 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f0b1cf-cb69-4b21-932e-2c8dc5707c61 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:48.118988 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cfbae1-9256-49e4-9080-8e3c35741fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:48.119654 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cfbae1-9256-49e4-9080-8e3c35741fc7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:48.119937 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cfbae1-9256-49e4-9080-8e3c35741fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:48.120211 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cfbae1-9256-49e4-9080-8e3c35741fc7 admin admin] PATH_INFO: `/v3/projects/2111959fe7e94015bd8f0383ab5c6f46/users/9039b37485b44800b2aff38093793ccb/roles/3f2ca33902a04b6dbc2ef7d06d7289d3` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:48.122327 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cfbae1-9256-49e4-9080-8e3c35741fc7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2111959fe7e94015bd8f0383ab5c6f46, user_id=9039b37485b44800b2aff38093793ccb, role_id=3f2ca33902a04b6dbc2ef7d06d7289d3)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:48.125951 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cfbae1-9256-49e4-9080-8e3c35741fc7 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:48.127143 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f0b1cf-cb69-4b21-932e-2c8dc5707c61 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:48.132909 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c350eb72-521e-4a51-ae83-856d1ac9a388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:48.133537 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c350eb72-521e-4a51-ae83-856d1ac9a388 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:48.133816 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c350eb72-521e-4a51-ae83-856d1ac9a388 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:48.134088 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c350eb72-521e-4a51-ae83-856d1ac9a388 admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:48.134714 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c350eb72-521e-4a51-ae83-856d1ac9a388 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:48.136762 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c350eb72-521e-4a51-ae83-856d1ac9a388 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:48.141719 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1071/2160] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:14:48 2022] PUT /identity/v3/projects/2111959fe7e94015bd8f0383ab5c6f46/users/9039b37485b44800b2aff38093793ccb/roles/3f2ca33902a04b6dbc2ef7d06d7289d3 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:14:48.176868 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1090/2161] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 9 12:14:48 2022] POST /identity/v3/projects => generated 379 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:14:48.182098 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cfbae1-9256-49e4-9080-8e3c35741fc7 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:48.264711 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cfbae1-9256-49e4-9080-8e3c35741fc7 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:48.278427 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:14:48.365103 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33ffa24-66ec-4c47-9e7f-5837670478e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:48.365698 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33ffa24-66ec-4c47-9e7f-5837670478e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:48.366869 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33ffa24-66ec-4c47-9e7f-5837670478e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:48.367425 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33ffa24-66ec-4c47-9e7f-5837670478e4 admin admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:48.368562 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33ffa24-66ec-4c47-9e7f-5837670478e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:48.371614 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33ffa24-66ec-4c47-9e7f-5837670478e4 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:48.442868 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1072/2162] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:14:48 2022] POST /identity/v3/users => generated 290 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:14:48.445020 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c350eb72-521e-4a51-ae83-856d1ac9a388 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:48.459977 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:14:48.465367 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c350eb72-521e-4a51-ae83-856d1ac9a388 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:48.473729 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a87c0a-5726-4234-9dd8-319d7d9516f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:48.474838 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a87c0a-5726-4234-9dd8-319d7d9516f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:48.475155 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a87c0a-5726-4234-9dd8-319d7d9516f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:48.476039 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a87c0a-5726-4234-9dd8-319d7d9516f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:48.476706 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a87c0a-5726-4234-9dd8-319d7d9516f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:48.479258 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a87c0a-5726-4234-9dd8-319d7d9516f0 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:48.493716 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1091/2163] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:14:48 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:14:48.495151 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33ffa24-66ec-4c47-9e7f-5837670478e4 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:48.507477 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33ffa24-66ec-4c47-9e7f-5837670478e4 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:48.512745 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6275b4c3-2108-482a-a13e-87ea21deead6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:48.513352 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6275b4c3-2108-482a-a13e-87ea21deead6 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:48.513626 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6275b4c3-2108-482a-a13e-87ea21deead6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:48.513892 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6275b4c3-2108-482a-a13e-87ea21deead6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:48.514554 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6275b4c3-2108-482a-a13e-87ea21deead6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:48.516212 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6275b4c3-2108-482a-a13e-87ea21deead6 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:48.530146 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1073/2164] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:14:48 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:14:48.535459 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a87c0a-5726-4234-9dd8-319d7d9516f0 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:48.550467 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a87c0a-5726-4234-9dd8-319d7d9516f0 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:48.561566 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557b378a-d938-4a14-bd35-5add049db926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:48.562371 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557b378a-d938-4a14-bd35-5add049db926 admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:48.562860 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557b378a-d938-4a14-bd35-5add049db926 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:48.563267 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557b378a-d938-4a14-bd35-5add049db926 admin admin] PATH_INFO: `/v3/projects/9a23ff2fa69e4a9db1d159b1c31bdeb4/users/f1b8e6f2bcdb4e1688d8d864add655a2/roles/eecd2f160dd844f5906f3d345780c501` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:48.582002 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557b378a-d938-4a14-bd35-5add049db926 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a23ff2fa69e4a9db1d159b1c31bdeb4, user_id=f1b8e6f2bcdb4e1688d8d864add655a2, role_id=eecd2f160dd844f5906f3d345780c501)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:48.584907 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557b378a-d938-4a14-bd35-5add049db926 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:48.608170 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1092/2165] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:14:48 2022] PUT /identity/v3/projects/9a23ff2fa69e4a9db1d159b1c31bdeb4/users/f1b8e6f2bcdb4e1688d8d864add655a2/roles/eecd2f160dd844f5906f3d345780c501 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:14:48.622044 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912574b8-1201-4107-aabb-f4ace07c58c0 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:48.622341 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912574b8-1201-4107-aabb-f4ace07c58c0 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:48.622621 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912574b8-1201-4107-aabb-f4ace07c58c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:48.670653 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-912574b8-1201-4107-aabb-f4ace07c58c0 None None] MFA Rules not processed for user `f1b8e6f2bcdb4e1688d8d864add655a2`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:14:48.674673 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-912574b8-1201-4107-aabb-f4ace07c58c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:48.746498 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1074/2166] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:14:48 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:14:48.751111 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557b378a-d938-4a14-bd35-5add049db926 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:48.821255 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557b378a-d938-4a14-bd35-5add049db926 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:48.826351 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065273b9-aeff-4f07-8a8c-ea8bb1b5c400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:48.827458 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065273b9-aeff-4f07-8a8c-ea8bb1b5c400 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:48.827458 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065273b9-aeff-4f07-8a8c-ea8bb1b5c400 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:48.827638 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065273b9-aeff-4f07-8a8c-ea8bb1b5c400 admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:48.828987 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065273b9-aeff-4f07-8a8c-ea8bb1b5c400 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:48.830157 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065273b9-aeff-4f07-8a8c-ea8bb1b5c400 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:48.864379 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1093/2167] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 9 12:14:48 2022] POST /identity/v3/projects => generated 381 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:14:48.869264 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912574b8-1201-4107-aabb-f4ace07c58c0 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:48.882713 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912574b8-1201-4107-aabb-f4ace07c58c0 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:48.956871 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef903f0-0705-4429-8b5c-bda49d87b7a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:48.957800 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef903f0-0705-4429-8b5c-bda49d87b7a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:48.958197 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef903f0-0705-4429-8b5c-bda49d87b7a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:48.958563 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef903f0-0705-4429-8b5c-bda49d87b7a7 admin admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:48.959491 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef903f0-0705-4429-8b5c-bda49d87b7a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:48.962092 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef903f0-0705-4429-8b5c-bda49d87b7a7 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:48.993312 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1075/2168] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:14:48 2022] POST /identity/v3/users => generated 297 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:14:48.997248 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065273b9-aeff-4f07-8a8c-ea8bb1b5c400 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:49.013393 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065273b9-aeff-4f07-8a8c-ea8bb1b5c400 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:49.026186 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3289407a-099d-458c-a0c2-195d3a05c23a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:49.026467 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3289407a-099d-458c-a0c2-195d3a05c23a admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:49.026748 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3289407a-099d-458c-a0c2-195d3a05c23a admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:49.027048 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3289407a-099d-458c-a0c2-195d3a05c23a admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:49.027711 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3289407a-099d-458c-a0c2-195d3a05c23a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:49.029322 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3289407a-099d-458c-a0c2-195d3a05c23a admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:49.057868 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1094/2169] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:14:48 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:14:49.059462 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef903f0-0705-4429-8b5c-bda49d87b7a7 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:49.074020 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef903f0-0705-4429-8b5c-bda49d87b7a7 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:49.082558 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb850eff-90db-4dd4-90a4-abaf20388949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:49.083371 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb850eff-90db-4dd4-90a4-abaf20388949 admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:49.083806 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb850eff-90db-4dd4-90a4-abaf20388949 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:49.084274 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb850eff-90db-4dd4-90a4-abaf20388949 admin admin] PATH_INFO: `/v3/projects/341553cd13894e039efa6503622a52e4/users/81dcc4467a0a4f889a3752b7b208ea36/roles/3f2ca33902a04b6dbc2ef7d06d7289d3` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:49.088192 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:14:49.104838 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb850eff-90db-4dd4-90a4-abaf20388949 admin admin] RBAC: Authorizing `identity:create_grant(project_id=341553cd13894e039efa6503622a52e4, user_id=81dcc4467a0a4f889a3752b7b208ea36, role_id=3f2ca33902a04b6dbc2ef7d06d7289d3)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:49.108026 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb850eff-90db-4dd4-90a4-abaf20388949 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:49.129805 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3289407a-099d-458c-a0c2-195d3a05c23a admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:49.130368 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1076/2170] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:14:49 2022] PUT /identity/v3/projects/341553cd13894e039efa6503622a52e4/users/81dcc4467a0a4f889a3752b7b208ea36/roles/3f2ca33902a04b6dbc2ef7d06d7289d3 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:14:49.141383 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3289407a-099d-458c-a0c2-195d3a05c23a admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:49.237854 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3215419a-9f33-405d-ba42-be2ee9b7e331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:49.238480 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3215419a-9f33-405d-ba42-be2ee9b7e331 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:49.238765 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3215419a-9f33-405d-ba42-be2ee9b7e331 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:49.239078 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3215419a-9f33-405d-ba42-be2ee9b7e331 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:49.239782 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3215419a-9f33-405d-ba42-be2ee9b7e331 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:49.242197 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3215419a-9f33-405d-ba42-be2ee9b7e331 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:49.253529 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1095/2171] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:14:49 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:14:49.257970 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb850eff-90db-4dd4-90a4-abaf20388949 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:49.269321 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb850eff-90db-4dd4-90a4-abaf20388949 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:49.277137 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a85aba2-a882-45f5-9bcf-f3f18f6b4d79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:49.277762 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a85aba2-a882-45f5-9bcf-f3f18f6b4d79 admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:49.278041 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a85aba2-a882-45f5-9bcf-f3f18f6b4d79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:49.278317 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a85aba2-a882-45f5-9bcf-f3f18f6b4d79 admin admin] PATH_INFO: `/v3/projects/341553cd13894e039efa6503622a52e4/users/81dcc4467a0a4f889a3752b7b208ea36/roles/3f2ca33902a04b6dbc2ef7d06d7289d3` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:49.280702 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a85aba2-a882-45f5-9bcf-f3f18f6b4d79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=341553cd13894e039efa6503622a52e4, user_id=81dcc4467a0a4f889a3752b7b208ea36, role_id=3f2ca33902a04b6dbc2ef7d06d7289d3)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:49.285385 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a85aba2-a882-45f5-9bcf-f3f18f6b4d79 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:49.299496 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1077/2172] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:14:49 2022] PUT /identity/v3/projects/341553cd13894e039efa6503622a52e4/users/81dcc4467a0a4f889a3752b7b208ea36/roles/3f2ca33902a04b6dbc2ef7d06d7289d3 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:14:49.312991 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5691149c-1c4d-405f-a777-b956cb3e236e None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:49.313284 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5691149c-1c4d-405f-a777-b956cb3e236e None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:49.313553 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5691149c-1c4d-405f-a777-b956cb3e236e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:49.358186 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-5691149c-1c4d-405f-a777-b956cb3e236e None None] MFA Rules not processed for user `81dcc4467a0a4f889a3752b7b208ea36`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:14:49.359964 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-5691149c-1c4d-405f-a777-b956cb3e236e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:49.430774 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:14:49.495270 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1096/2173] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:14:49 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:14:49.506192 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a85aba2-a882-45f5-9bcf-f3f18f6b4d79 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:49.508933 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-5a85aba2-a882-45f5-9bcf-f3f18f6b4d79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:49.663399 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a85aba2-a882-45f5-9bcf-f3f18f6b4d79 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:49.682032 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dc570a-5ba2-47c9-8a4a-d756728f5600 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535c258100df40b58a5ee44b2c21d621', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:49.685510 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dc570a-5ba2-47c9-8a4a-d756728f5600 service cinder] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:49.686011 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dc570a-5ba2-47c9-8a4a-d756728f5600 service cinder] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:49.686623 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dc570a-5ba2-47c9-8a4a-d756728f5600 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:49.717782 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dc570a-5ba2-47c9-8a4a-d756728f5600 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:49.719288 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dc570a-5ba2-47c9-8a4a-d756728f5600 service cinder] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:49.740759 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1078/2174] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:14:49 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:14:50.526155 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5691149c-1c4d-405f-a777-b956cb3e236e None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:50.651440 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5691149c-1c4d-405f-a777-b956cb3e236e None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:50.669484 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3f43af-6f20-4e8a-8078-1c1510f29bd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:50.674431 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3f43af-6f20-4e8a-8078-1c1510f29bd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:50.674751 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3f43af-6f20-4e8a-8078-1c1510f29bd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:50.675066 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3f43af-6f20-4e8a-8078-1c1510f29bd6 admin admin] PATH_INFO: `/v3/users/f1b8e6f2bcdb4e1688d8d864add655a2` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:50.689888 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3f43af-6f20-4e8a-8078-1c1510f29bd6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1b8e6f2bcdb4e1688d8d864add655a2)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:50.699967 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3f43af-6f20-4e8a-8078-1c1510f29bd6 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:50.864627 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-5e3f43af-6f20-4e8a-8078-1c1510f29bd6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1b8e6f2bcdb4e1688d8d864add655a2'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:50.889199 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-5e3f43af-6f20-4e8a-8078-1c1510f29bd6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1b8e6f2bcdb4e1688d8d864add655a2'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:50.918940 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-5e3f43af-6f20-4e8a-8078-1c1510f29bd6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1b8e6f2bcdb4e1688d8d864add655a2'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:50.946505 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-5e3f43af-6f20-4e8a-8078-1c1510f29bd6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1b8e6f2bcdb4e1688d8d864add655a2'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:50.947564 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-5e3f43af-6f20-4e8a-8078-1c1510f29bd6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1b8e6f2bcdb4e1688d8d864add655a2'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:50.964697 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1097/2175] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 9 12:14:50 2022] DELETE /identity/v3/users/f1b8e6f2bcdb4e1688d8d864add655a2 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:14:50.968916 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c696c39-f400-47a8-af49-b0cac45d3b18 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:50.969534 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c696c39-f400-47a8-af49-b0cac45d3b18 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:50.969534 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c696c39-f400-47a8-af49-b0cac45d3b18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:51.004314 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-2c696c39-f400-47a8-af49-b0cac45d3b18 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:14:51.005550 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-2c696c39-f400-47a8-af49-b0cac45d3b18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:51.084457 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1079/2176] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:14:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:14:51.094258 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3f43af-6f20-4e8a-8078-1c1510f29bd6 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:51.095800 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-5e3f43af-6f20-4e8a-8078-1c1510f29bd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:51.164849 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:14:51.192967 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3f43af-6f20-4e8a-8078-1c1510f29bd6 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:51.198292 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8724e452-59c8-4c8f-bc3a-c1baf5afe4fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:51.198683 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8724e452-59c8-4c8f-bc3a-c1baf5afe4fb service neutron] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:51.198988 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8724e452-59c8-4c8f-bc3a-c1baf5afe4fb service neutron] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:51.199256 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8724e452-59c8-4c8f-bc3a-c1baf5afe4fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:51.209888 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8724e452-59c8-4c8f-bc3a-c1baf5afe4fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:51.213780 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8724e452-59c8-4c8f-bc3a-c1baf5afe4fb service neutron] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:51.230448 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1098/2177] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:14:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:14:51.646267 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c696c39-f400-47a8-af49-b0cac45d3b18 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:51.648304 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-2c696c39-f400-47a8-af49-b0cac45d3b18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:51.677303 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c696c39-f400-47a8-af49-b0cac45d3b18 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:51.689674 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3852265e-3665-491b-b27a-69774104ef43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:51.690331 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3852265e-3665-491b-b27a-69774104ef43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:51.690609 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3852265e-3665-491b-b27a-69774104ef43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:51.690881 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3852265e-3665-491b-b27a-69774104ef43 admin admin] PATH_INFO: `/v3/projects/9a23ff2fa69e4a9db1d159b1c31bdeb4` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:51.695757 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3852265e-3665-491b-b27a-69774104ef43 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a23ff2fa69e4a9db1d159b1c31bdeb4)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:51.697717 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3852265e-3665-491b-b27a-69774104ef43 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:51.792842 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-3852265e-3665-491b-b27a-69774104ef43 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a23ff2fa69e4a9db1d159b1c31bdeb4', 'request_id': 'req-3852265e-3665-491b-b27a-69774104ef43'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:51.822503 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-3852265e-3665-491b-b27a-69774104ef43 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a23ff2fa69e4a9db1d159b1c31bdeb4', 'request_id': 'req-3852265e-3665-491b-b27a-69774104ef43'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:51.839715 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-3852265e-3665-491b-b27a-69774104ef43 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a23ff2fa69e4a9db1d159b1c31bdeb4', 'request_id': 'req-3852265e-3665-491b-b27a-69774104ef43'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:51.850076 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:14:51.859517 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-3852265e-3665-491b-b27a-69774104ef43 admin admin] The token cache is being invalidate because project 9a23ff2fa69e4a9db1d159b1c31bdeb4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:14:51.859763 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-3852265e-3665-491b-b27a-69774104ef43 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:51.860546 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-3852265e-3665-491b-b27a-69774104ef43 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:51.862494 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1080/2178] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 9 12:14:51 2022] DELETE /identity/v3/projects/9a23ff2fa69e4a9db1d159b1c31bdeb4 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:14:51.867540 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8724e452-59c8-4c8f-bc3a-c1baf5afe4fb service neutron] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:51.881440 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8724e452-59c8-4c8f-bc3a-c1baf5afe4fb service neutron] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:52.058092 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5cd73f-16fd-481a-b972-cd1cee86c259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:52.058740 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5cd73f-16fd-481a-b972-cd1cee86c259 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:52.059304 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5cd73f-16fd-481a-b972-cd1cee86c259 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:52.059575 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5cd73f-16fd-481a-b972-cd1cee86c259 admin admin] PATH_INFO: `/v3/users/81dcc4467a0a4f889a3752b7b208ea36` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:52.062726 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5cd73f-16fd-481a-b972-cd1cee86c259 admin admin] RBAC: Authorizing `identity:delete_user(user_id=81dcc4467a0a4f889a3752b7b208ea36)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:52.064810 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5cd73f-16fd-481a-b972-cd1cee86c259 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:52.180099 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-8b5cd73f-16fd-481a-b972-cd1cee86c259 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81dcc4467a0a4f889a3752b7b208ea36'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:52.194556 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-8b5cd73f-16fd-481a-b972-cd1cee86c259 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81dcc4467a0a4f889a3752b7b208ea36'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:52.211563 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-8b5cd73f-16fd-481a-b972-cd1cee86c259 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81dcc4467a0a4f889a3752b7b208ea36'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:52.230250 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-8b5cd73f-16fd-481a-b972-cd1cee86c259 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81dcc4467a0a4f889a3752b7b208ea36'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:52.235060 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-8b5cd73f-16fd-481a-b972-cd1cee86c259 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81dcc4467a0a4f889a3752b7b208ea36'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:52.255685 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1099/2179] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 9 12:14:51 2022] DELETE /identity/v3/users/81dcc4467a0a4f889a3752b7b208ea36 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:14:52.766832 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3852265e-3665-491b-b27a-69774104ef43 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:52.768333 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-3852265e-3665-491b-b27a-69774104ef43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:52.797640 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3852265e-3665-491b-b27a-69774104ef43 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:52.879825 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9590aa2-0b8a-4f02-934b-389ff32b79af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:52.881078 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9590aa2-0b8a-4f02-934b-389ff32b79af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:52.881078 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9590aa2-0b8a-4f02-934b-389ff32b79af admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:52.881078 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9590aa2-0b8a-4f02-934b-389ff32b79af admin admin] PATH_INFO: `/v3/projects/341553cd13894e039efa6503622a52e4` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:52.882410 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9590aa2-0b8a-4f02-934b-389ff32b79af admin admin] RBAC: Authorizing `identity:delete_project(project_id=341553cd13894e039efa6503622a52e4)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:52.884431 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9590aa2-0b8a-4f02-934b-389ff32b79af admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:52.955936 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-e9590aa2-0b8a-4f02-934b-389ff32b79af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '341553cd13894e039efa6503622a52e4', 'request_id': 'req-e9590aa2-0b8a-4f02-934b-389ff32b79af'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:52.961982 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-e9590aa2-0b8a-4f02-934b-389ff32b79af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '341553cd13894e039efa6503622a52e4', 'request_id': 'req-e9590aa2-0b8a-4f02-934b-389ff32b79af'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:52.971841 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-e9590aa2-0b8a-4f02-934b-389ff32b79af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '341553cd13894e039efa6503622a52e4', 'request_id': 'req-e9590aa2-0b8a-4f02-934b-389ff32b79af'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:52.992776 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-e9590aa2-0b8a-4f02-934b-389ff32b79af admin admin] The token cache is being invalidate because project 341553cd13894e039efa6503622a52e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:14:52.992776 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-e9590aa2-0b8a-4f02-934b-389ff32b79af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:52.993602 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-e9590aa2-0b8a-4f02-934b-389ff32b79af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:14:52.999339 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1081/2180] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 9 12:14:52 2022] DELETE /identity/v3/projects/341553cd13894e039efa6503622a52e4 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:14:53.006072 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b225d3da-81e6-4483-a1d0-dc34a5327037 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:53.006602 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b225d3da-81e6-4483-a1d0-dc34a5327037 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:53.006768 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b225d3da-81e6-4483-a1d0-dc34a5327037 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:53.031786 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-b225d3da-81e6-4483-a1d0-dc34a5327037 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:14:53.033065 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-b225d3da-81e6-4483-a1d0-dc34a5327037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:53.168698 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1100/2181] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:14:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:14:53.169949 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1b286e-4de3-413a-b238-4c5864981ea1 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:53.170288 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1b286e-4de3-413a-b238-4c5864981ea1 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:53.170553 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1b286e-4de3-413a-b238-4c5864981ea1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:53.210868 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-3c1b286e-4de3-413a-b238-4c5864981ea1 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:14:53.212395 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-3c1b286e-4de3-413a-b238-4c5864981ea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:53.221838 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1082/2182] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:14:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:14:53.226539 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b225d3da-81e6-4483-a1d0-dc34a5327037 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:53.246259 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b225d3da-81e6-4483-a1d0-dc34a5327037 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:53.253600 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cb1012-b972-48d0-bb57-5b4d599f016c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:53.254522 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cb1012-b972-48d0-bb57-5b4d599f016c admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:53.255079 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cb1012-b972-48d0-bb57-5b4d599f016c admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:53.257751 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cb1012-b972-48d0-bb57-5b4d599f016c admin admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:53.258687 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cb1012-b972-48d0-bb57-5b4d599f016c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:14:53.259239 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cb1012-b972-48d0-bb57-5b4d599f016c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:53.268041 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cb1012-b972-48d0-bb57-5b4d599f016c admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:53.271071 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:14:53.290565 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1101/2183] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 9 12:14:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:14:53.292213 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1b286e-4de3-413a-b238-4c5864981ea1 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:53.304962 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1b286e-4de3-413a-b238-4c5864981ea1 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:53.310767 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c1e46f-a5d6-44fc-b416-0b1773b44661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:53.311797 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c1e46f-a5d6-44fc-b416-0b1773b44661 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:53.312292 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c1e46f-a5d6-44fc-b416-0b1773b44661 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:53.313035 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c1e46f-a5d6-44fc-b416-0b1773b44661 admin admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:53.313986 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c1e46f-a5d6-44fc-b416-0b1773b44661 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:53.316196 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c1e46f-a5d6-44fc-b416-0b1773b44661 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:53.364014 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1083/2184] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 9 12:14:53 2022] POST /identity/v3/projects => generated 367 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:14:53.368826 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cb1012-b972-48d0-bb57-5b4d599f016c admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:53.387538 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cb1012-b972-48d0-bb57-5b4d599f016c admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:53.505734 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501201de-2121-4ea6-ad57-d1f7bbbd43f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:53.507095 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501201de-2121-4ea6-ad57-d1f7bbbd43f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:53.507495 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501201de-2121-4ea6-ad57-d1f7bbbd43f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:53.507781 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501201de-2121-4ea6-ad57-d1f7bbbd43f6 admin admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:53.508614 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501201de-2121-4ea6-ad57-d1f7bbbd43f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:53.510541 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501201de-2121-4ea6-ad57-d1f7bbbd43f6 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:53.551630 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1102/2185] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:14:53 2022] POST /identity/v3/users => generated 284 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:14:53.555758 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c1e46f-a5d6-44fc-b416-0b1773b44661 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:53.569727 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c1e46f-a5d6-44fc-b416-0b1773b44661 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:53.585440 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b04995-9ef9-4c46-b60e-d110d0cbd287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:53.586078 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b04995-9ef9-4c46-b60e-d110d0cbd287 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:53.586353 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b04995-9ef9-4c46-b60e-d110d0cbd287 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:53.586764 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b04995-9ef9-4c46-b60e-d110d0cbd287 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:53.587455 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b04995-9ef9-4c46-b60e-d110d0cbd287 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:53.589077 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b04995-9ef9-4c46-b60e-d110d0cbd287 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:53.611001 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1084/2186] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:14:53 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 12:14:53.614139 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501201de-2121-4ea6-ad57-d1f7bbbd43f6 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:53.630283 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501201de-2121-4ea6-ad57-d1f7bbbd43f6 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:53.639045 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb494c1-433a-4f5d-ae10-cce1fd670b53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:53.640028 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb494c1-433a-4f5d-ae10-cce1fd670b53 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:53.640372 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb494c1-433a-4f5d-ae10-cce1fd670b53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:53.640821 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb494c1-433a-4f5d-ae10-cce1fd670b53 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:53.642688 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb494c1-433a-4f5d-ae10-cce1fd670b53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:53.649267 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb494c1-433a-4f5d-ae10-cce1fd670b53 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:53.666186 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1103/2187] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:14:53 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:14:53.667233 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b04995-9ef9-4c46-b60e-d110d0cbd287 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:53.679120 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b04995-9ef9-4c46-b60e-d110d0cbd287 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:53.685042 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3901627-d160-4430-9a1d-e74cd3b13b26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:53.685883 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3901627-d160-4430-9a1d-e74cd3b13b26 admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:53.686337 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3901627-d160-4430-9a1d-e74cd3b13b26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:53.686615 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3901627-d160-4430-9a1d-e74cd3b13b26 admin admin] PATH_INFO: `/v3/projects/46377e6c51234e3991205d996ea0ef5a/users/324c01ff5dc24f75b74a61e7da3a5c42/roles/eecd2f160dd844f5906f3d345780c501` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:53.707566 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3901627-d160-4430-9a1d-e74cd3b13b26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=46377e6c51234e3991205d996ea0ef5a, user_id=324c01ff5dc24f75b74a61e7da3a5c42, role_id=eecd2f160dd844f5906f3d345780c501)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:53.711913 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3901627-d160-4430-9a1d-e74cd3b13b26 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:53.737008 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1085/2188] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:14:53 2022] PUT /identity/v3/projects/46377e6c51234e3991205d996ea0ef5a/users/324c01ff5dc24f75b74a61e7da3a5c42/roles/eecd2f160dd844f5906f3d345780c501 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:14:53.750474 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4f0f4d-c1c3-41ae-9d45-bba189b6e3a0 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:53.750793 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4f0f4d-c1c3-41ae-9d45-bba189b6e3a0 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:53.751342 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4f0f4d-c1c3-41ae-9d45-bba189b6e3a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:53.795432 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-fe4f0f4d-c1c3-41ae-9d45-bba189b6e3a0 None None] MFA Rules not processed for user `324c01ff5dc24f75b74a61e7da3a5c42`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:14:53.796779 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-fe4f0f4d-c1c3-41ae-9d45-bba189b6e3a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:53.880374 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1104/2189] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:14:53 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:14:53.889537 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3901627-d160-4430-9a1d-e74cd3b13b26 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:53.891549 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-b3901627-d160-4430-9a1d-e74cd3b13b26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:53.923776 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:14:53.964807 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3901627-d160-4430-9a1d-e74cd3b13b26 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:53.968315 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c041910e-6ef5-4d23-9a5f-6793c5cd2ef3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37c4e14aa8e04d3e85a406b9363bca2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:53.969424 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c041910e-6ef5-4d23-9a5f-6793c5cd2ef3 service glance] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:53.969424 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c041910e-6ef5-4d23-9a5f-6793c5cd2ef3 service glance] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:53.969613 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c041910e-6ef5-4d23-9a5f-6793c5cd2ef3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:53.979374 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c041910e-6ef5-4d23-9a5f-6793c5cd2ef3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:53.980738 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c041910e-6ef5-4d23-9a5f-6793c5cd2ef3 service glance] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:53.994856 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1086/2190] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:14:53 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:14:54.269254 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8938a7df-c854-4442-b3ab-135f2543c2a1 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:54.269806 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8938a7df-c854-4442-b3ab-135f2543c2a1 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:54.269806 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8938a7df-c854-4442-b3ab-135f2543c2a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:54.309953 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-8938a7df-c854-4442-b3ab-135f2543c2a1 None None] MFA Rules not processed for user `9039b37485b44800b2aff38093793ccb`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:14:54.311803 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-8938a7df-c854-4442-b3ab-135f2543c2a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:54.392248 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:14:54.428203 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1105/2191] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:14:54 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:14:54.443200 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c041910e-6ef5-4d23-9a5f-6793c5cd2ef3 service glance] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:14:54.445535 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-c041910e-6ef5-4d23-9a5f-6793c5cd2ef3 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:54.572811 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c041910e-6ef5-4d23-9a5f-6793c5cd2ef3 service glance] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:14:54.578286 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1442de25-100c-414c-8170-fa7dc7835422 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70bac169d5cd43e1926f310344ac11fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:14:54.578991 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1442de25-100c-414c-8170-fa7dc7835422 service nova] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:54.579301 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1442de25-100c-414c-8170-fa7dc7835422 service nova] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:54.579542 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1442de25-100c-414c-8170-fa7dc7835422 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:54.581595 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-1442de25-100c-414c-8170-fa7dc7835422 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:54.680033 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1442de25-100c-414c-8170-fa7dc7835422 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:14:54.697513 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1442de25-100c-414c-8170-fa7dc7835422 service nova] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:14:54.710432 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1087/2192] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:14:54 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:14:56.304297 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77767b4-b641-4c41-8dab-bc15bd473ab6 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:14:56.304968 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77767b4-b641-4c41-8dab-bc15bd473ab6 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:14:56.304968 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77767b4-b641-4c41-8dab-bc15bd473ab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:14:56.326411 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-d77767b4-b641-4c41-8dab-bc15bd473ab6 None None] MFA Rules not processed for user `9750f671d674447f9fbf36d382573c37`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:14:56.327841 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-d77767b4-b641-4c41-8dab-bc15bd473ab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:14:56.335430 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1106/2193] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:14:56 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:15:00.490457 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1442de25-100c-414c-8170-fa7dc7835422 service nova] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:00.491964 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-1442de25-100c-414c-8170-fa7dc7835422 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:15:00.573950 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1442de25-100c-414c-8170-fa7dc7835422 service nova] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:00.579646 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bca9ddf-9e28-467c-9201-027e038e527c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:00.580318 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bca9ddf-9e28-467c-9201-027e038e527c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:00.581069 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bca9ddf-9e28-467c-9201-027e038e527c admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:00.581069 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bca9ddf-9e28-467c-9201-027e038e527c admin admin] PATH_INFO: `/v3/users/da1c144d866b40fd88f1dc017ee8dcb1` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:00.581993 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bca9ddf-9e28-467c-9201-027e038e527c admin admin] RBAC: Authorizing `identity:delete_user(user_id=da1c144d866b40fd88f1dc017ee8dcb1)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:00.584066 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bca9ddf-9e28-467c-9201-027e038e527c admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:00.592847 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:15:00.649526 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-4bca9ddf-9e28-467c-9201-027e038e527c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da1c144d866b40fd88f1dc017ee8dcb1'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:00.665490 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-4bca9ddf-9e28-467c-9201-027e038e527c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da1c144d866b40fd88f1dc017ee8dcb1'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:00.673338 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-4bca9ddf-9e28-467c-9201-027e038e527c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da1c144d866b40fd88f1dc017ee8dcb1'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:00.681450 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-4bca9ddf-9e28-467c-9201-027e038e527c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da1c144d866b40fd88f1dc017ee8dcb1'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:00.682271 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-4bca9ddf-9e28-467c-9201-027e038e527c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da1c144d866b40fd88f1dc017ee8dcb1'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:00.688955 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1088/2194] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 9 12:15:00 2022] DELETE /identity/v3/users/da1c144d866b40fd88f1dc017ee8dcb1 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:15:00.694569 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cd6a59-725f-4d7b-bfc9-6e16096100d3 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:00.695134 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cd6a59-725f-4d7b-bfc9-6e16096100d3 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:00.695134 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cd6a59-725f-4d7b-bfc9-6e16096100d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:00.715720 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-81cd6a59-725f-4d7b-bfc9-6e16096100d3 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:15:00.716940 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-81cd6a59-725f-4d7b-bfc9-6e16096100d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:15:00.783868 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1107/2195] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:15:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:15:00.794834 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bca9ddf-9e28-467c-9201-027e038e527c admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:00.796390 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-4bca9ddf-9e28-467c-9201-027e038e527c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:15:00.858082 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bca9ddf-9e28-467c-9201-027e038e527c admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:00.862735 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a389a1c-276e-4aa2-af69-8e053449d165 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:00.863426 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a389a1c-276e-4aa2-af69-8e053449d165 service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:00.863704 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a389a1c-276e-4aa2-af69-8e053449d165 service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:00.864002 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a389a1c-276e-4aa2-af69-8e053449d165 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:00.873717 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a389a1c-276e-4aa2-af69-8e053449d165 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:00.875178 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a389a1c-276e-4aa2-af69-8e053449d165 service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:00.884727 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1089/2196] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:15:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:15:01.199843 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cd6a59-725f-4d7b-bfc9-6e16096100d3 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:01.201333 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-81cd6a59-725f-4d7b-bfc9-6e16096100d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:15:01.215441 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cd6a59-725f-4d7b-bfc9-6e16096100d3 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:01.220078 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4b2905-4402-4c69-a0fb-eeef220a0635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:01.220714 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4b2905-4402-4c69-a0fb-eeef220a0635 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:01.221067 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4b2905-4402-4c69-a0fb-eeef220a0635 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:01.221518 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4b2905-4402-4c69-a0fb-eeef220a0635 admin admin] PATH_INFO: `/v3/projects/cdab9b7a5d6645ad8b03bebdb3ca1b24` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:01.223943 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4b2905-4402-4c69-a0fb-eeef220a0635 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cdab9b7a5d6645ad8b03bebdb3ca1b24)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:01.226444 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4b2905-4402-4c69-a0fb-eeef220a0635 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:01.278415 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-1d4b2905-4402-4c69-a0fb-eeef220a0635 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cdab9b7a5d6645ad8b03bebdb3ca1b24', 'request_id': 'req-1d4b2905-4402-4c69-a0fb-eeef220a0635'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:01.284466 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-1d4b2905-4402-4c69-a0fb-eeef220a0635 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cdab9b7a5d6645ad8b03bebdb3ca1b24', 'request_id': 'req-1d4b2905-4402-4c69-a0fb-eeef220a0635'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:01.292292 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-1d4b2905-4402-4c69-a0fb-eeef220a0635 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cdab9b7a5d6645ad8b03bebdb3ca1b24', 'request_id': 'req-1d4b2905-4402-4c69-a0fb-eeef220a0635'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:01.311175 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-1d4b2905-4402-4c69-a0fb-eeef220a0635 admin admin] The token cache is being invalidate because project cdab9b7a5d6645ad8b03bebdb3ca1b24 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81591) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:15:01.311467 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-1d4b2905-4402-4c69-a0fb-eeef220a0635 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:01.312131 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-1d4b2905-4402-4c69-a0fb-eeef220a0635 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:01.313625 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1108/2197] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 9 12:15:01 2022] DELETE /identity/v3/projects/cdab9b7a5d6645ad8b03bebdb3ca1b24 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:15:01.317510 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a389a1c-276e-4aa2-af69-8e053449d165 service neutron] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:01.328734 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a389a1c-276e-4aa2-af69-8e053449d165 service neutron] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:01.387806 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca12f37-cd47-4b2e-a9d1-1793c4afe134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:01.388458 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca12f37-cd47-4b2e-a9d1-1793c4afe134 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:01.388747 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca12f37-cd47-4b2e-a9d1-1793c4afe134 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:01.389004 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca12f37-cd47-4b2e-a9d1-1793c4afe134 admin admin] PATH_INFO: `/v3/users/fae3216365774960955aa5e31b2d3ebb` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:01.390145 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca12f37-cd47-4b2e-a9d1-1793c4afe134 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fae3216365774960955aa5e31b2d3ebb)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:01.392209 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca12f37-cd47-4b2e-a9d1-1793c4afe134 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:01.425164 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:15:01.445601 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-5ca12f37-cd47-4b2e-a9d1-1793c4afe134 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fae3216365774960955aa5e31b2d3ebb'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:01.457842 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-5ca12f37-cd47-4b2e-a9d1-1793c4afe134 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fae3216365774960955aa5e31b2d3ebb'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:01.466259 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-5ca12f37-cd47-4b2e-a9d1-1793c4afe134 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fae3216365774960955aa5e31b2d3ebb'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:01.478518 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-5ca12f37-cd47-4b2e-a9d1-1793c4afe134 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fae3216365774960955aa5e31b2d3ebb'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:01.479208 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-5ca12f37-cd47-4b2e-a9d1-1793c4afe134 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fae3216365774960955aa5e31b2d3ebb'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:01.485400 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1090/2198] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 9 12:15:01 2022] DELETE /identity/v3/users/fae3216365774960955aa5e31b2d3ebb => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:15:01.786991 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4b2905-4402-4c69-a0fb-eeef220a0635 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:01.788587 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-1d4b2905-4402-4c69-a0fb-eeef220a0635 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:15:01.808287 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4b2905-4402-4c69-a0fb-eeef220a0635 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:01.868001 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f73cde0-89fc-476e-b9b6-bde9a61d82d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:01.868885 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f73cde0-89fc-476e-b9b6-bde9a61d82d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:01.869172 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f73cde0-89fc-476e-b9b6-bde9a61d82d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:01.869436 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f73cde0-89fc-476e-b9b6-bde9a61d82d8 admin admin] PATH_INFO: `/v3/projects/2d091d9f3287438d81ee47dee3bef467` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:01.870546 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f73cde0-89fc-476e-b9b6-bde9a61d82d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d091d9f3287438d81ee47dee3bef467)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:01.872975 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f73cde0-89fc-476e-b9b6-bde9a61d82d8 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:01.879294 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:15:01.923712 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-2f73cde0-89fc-476e-b9b6-bde9a61d82d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d091d9f3287438d81ee47dee3bef467', 'request_id': 'req-2f73cde0-89fc-476e-b9b6-bde9a61d82d8'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:01.929905 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-2f73cde0-89fc-476e-b9b6-bde9a61d82d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d091d9f3287438d81ee47dee3bef467', 'request_id': 'req-2f73cde0-89fc-476e-b9b6-bde9a61d82d8'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:01.936413 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-2f73cde0-89fc-476e-b9b6-bde9a61d82d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d091d9f3287438d81ee47dee3bef467', 'request_id': 'req-2f73cde0-89fc-476e-b9b6-bde9a61d82d8'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:01.949476 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-2f73cde0-89fc-476e-b9b6-bde9a61d82d8 admin admin] The token cache is being invalidate because project 2d091d9f3287438d81ee47dee3bef467 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81591) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:15:01.949760 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-2f73cde0-89fc-476e-b9b6-bde9a61d82d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:01.950396 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-2f73cde0-89fc-476e-b9b6-bde9a61d82d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:01.951923 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1109/2199] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 9 12:15:01 2022] DELETE /identity/v3/projects/2d091d9f3287438d81ee47dee3bef467 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:15:01.956734 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1e20da-f570-4088-a1b9-def9f97e1d93 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:01.957220 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1e20da-f570-4088-a1b9-def9f97e1d93 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:01.957315 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1e20da-f570-4088-a1b9-def9f97e1d93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:01.979333 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-cf1e20da-f570-4088-a1b9-def9f97e1d93 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:15:01.980533 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-cf1e20da-f570-4088-a1b9-def9f97e1d93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:15:02.050585 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1091/2200] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:15:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:15:02.059529 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4b9577-b0fd-4192-96d7-d3cb909af5c0 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:02.059834 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4b9577-b0fd-4192-96d7-d3cb909af5c0 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:02.060128 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4b9577-b0fd-4192-96d7-d3cb909af5c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:02.082253 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-4e4b9577-b0fd-4192-96d7-d3cb909af5c0 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:15:02.083552 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-4e4b9577-b0fd-4192-96d7-d3cb909af5c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:15:02.093223 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1110/2201] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:15:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:15:02.097422 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1e20da-f570-4088-a1b9-def9f97e1d93 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:02.107778 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1e20da-f570-4088-a1b9-def9f97e1d93 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:02.112452 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef091eeb-8003-48a7-aca6-5adc0c6cbcf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:02.113099 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef091eeb-8003-48a7-aca6-5adc0c6cbcf0 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:02.113418 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef091eeb-8003-48a7-aca6-5adc0c6cbcf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:02.113694 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef091eeb-8003-48a7-aca6-5adc0c6cbcf0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:02.114377 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef091eeb-8003-48a7-aca6-5adc0c6cbcf0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:15:02.114674 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef091eeb-8003-48a7-aca6-5adc0c6cbcf0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:02.117217 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef091eeb-8003-48a7-aca6-5adc0c6cbcf0 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:02.129647 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1092/2202] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 9 12:15:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:15:02.134773 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4b9577-b0fd-4192-96d7-d3cb909af5c0 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:02.144847 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4b9577-b0fd-4192-96d7-d3cb909af5c0 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:02.151521 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4b82c2-4ef7-4927-96e9-75d2df0fb94f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:02.152141 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4b82c2-4ef7-4927-96e9-75d2df0fb94f admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:02.152456 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4b82c2-4ef7-4927-96e9-75d2df0fb94f admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:02.152681 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4b82c2-4ef7-4927-96e9-75d2df0fb94f admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:02.153323 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4b82c2-4ef7-4927-96e9-75d2df0fb94f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:02.155300 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4b82c2-4ef7-4927-96e9-75d2df0fb94f admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:02.204402 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1111/2203] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 9 12:15:02 2022] POST /identity/v3/projects => generated 393 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:15:02.208539 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef091eeb-8003-48a7-aca6-5adc0c6cbcf0 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:02.221831 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef091eeb-8003-48a7-aca6-5adc0c6cbcf0 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:02.290863 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa20a3e-56dc-42d8-a549-afd6157906cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:02.291922 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa20a3e-56dc-42d8-a549-afd6157906cc admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:02.292202 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa20a3e-56dc-42d8-a549-afd6157906cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:02.292472 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa20a3e-56dc-42d8-a549-afd6157906cc admin admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:02.293115 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa20a3e-56dc-42d8-a549-afd6157906cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:02.295154 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa20a3e-56dc-42d8-a549-afd6157906cc admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:02.322387 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1093/2204] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:15:02 2022] POST /identity/v3/users => generated 297 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:15:02.323677 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4b82c2-4ef7-4927-96e9-75d2df0fb94f admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:02.334633 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4b82c2-4ef7-4927-96e9-75d2df0fb94f admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:02.339341 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd5cedd-b578-4cbc-8558-2465d6533d68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:02.339935 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd5cedd-b578-4cbc-8558-2465d6533d68 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:02.340210 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd5cedd-b578-4cbc-8558-2465d6533d68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:02.340482 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd5cedd-b578-4cbc-8558-2465d6533d68 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:02.341118 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd5cedd-b578-4cbc-8558-2465d6533d68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:02.342878 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd5cedd-b578-4cbc-8558-2465d6533d68 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:02.357536 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1112/2205] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:15:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 12:15:02.360134 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa20a3e-56dc-42d8-a549-afd6157906cc admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:02.372501 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa20a3e-56dc-42d8-a549-afd6157906cc admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:02.377098 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2209f34e-4e77-48dd-b990-f165644dedd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:02.377691 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2209f34e-4e77-48dd-b990-f165644dedd5 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:02.377964 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2209f34e-4e77-48dd-b990-f165644dedd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:02.378232 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2209f34e-4e77-48dd-b990-f165644dedd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:02.378865 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2209f34e-4e77-48dd-b990-f165644dedd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:02.380530 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2209f34e-4e77-48dd-b990-f165644dedd5 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:02.391285 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1094/2206] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:15:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:15:02.395879 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd5cedd-b578-4cbc-8558-2465d6533d68 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:02.409891 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd5cedd-b578-4cbc-8558-2465d6533d68 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:02.415644 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483eb8f5-4996-48d5-90ee-bf6276a338e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:02.416478 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483eb8f5-4996-48d5-90ee-bf6276a338e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:02.416761 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483eb8f5-4996-48d5-90ee-bf6276a338e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:02.417032 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483eb8f5-4996-48d5-90ee-bf6276a338e0 admin admin] PATH_INFO: `/v3/projects/765a0a2ca4094dc1bcc5fedb201b2b16/users/addf4f48d9c843b8b7daca371a842f6c/roles/eecd2f160dd844f5906f3d345780c501` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:02.434972 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483eb8f5-4996-48d5-90ee-bf6276a338e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=765a0a2ca4094dc1bcc5fedb201b2b16, user_id=addf4f48d9c843b8b7daca371a842f6c, role_id=eecd2f160dd844f5906f3d345780c501)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:02.440600 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483eb8f5-4996-48d5-90ee-bf6276a338e0 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:02.458169 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1113/2207] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:15:02 2022] PUT /identity/v3/projects/765a0a2ca4094dc1bcc5fedb201b2b16/users/addf4f48d9c843b8b7daca371a842f6c/roles/eecd2f160dd844f5906f3d345780c501 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:15:02.470343 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e222f7a8-8dbf-4707-b8aa-1b4a26b3fc31 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:02.470856 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e222f7a8-8dbf-4707-b8aa-1b4a26b3fc31 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:02.471343 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e222f7a8-8dbf-4707-b8aa-1b4a26b3fc31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:02.491903 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:15:02.501051 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-e222f7a8-8dbf-4707-b8aa-1b4a26b3fc31 None None] MFA Rules not processed for user `addf4f48d9c843b8b7daca371a842f6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:15:02.503127 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-e222f7a8-8dbf-4707-b8aa-1b4a26b3fc31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:15:02.597538 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1095/2208] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:15:02 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:15:02.602380 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483eb8f5-4996-48d5-90ee-bf6276a338e0 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:02.687425 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483eb8f5-4996-48d5-90ee-bf6276a338e0 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:02.697504 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1427b7-1196-43a1-9647-7654fcbe41c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:02.698133 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1427b7-1196-43a1-9647-7654fcbe41c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:02.698413 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1427b7-1196-43a1-9647-7654fcbe41c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:02.698684 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1427b7-1196-43a1-9647-7654fcbe41c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:02.699360 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1427b7-1196-43a1-9647-7654fcbe41c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:02.701311 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1427b7-1196-43a1-9647-7654fcbe41c0 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:02.738490 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1114/2209] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 9 12:15:02 2022] POST /identity/v3/projects => generated 389 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:15:02.743119 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e222f7a8-8dbf-4707-b8aa-1b4a26b3fc31 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:02.754505 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e222f7a8-8dbf-4707-b8aa-1b4a26b3fc31 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:02.859856 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536fb176-feb7-4a7b-99ec-02319a286260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:02.860737 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536fb176-feb7-4a7b-99ec-02319a286260 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:02.860925 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536fb176-feb7-4a7b-99ec-02319a286260 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:02.861204 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536fb176-feb7-4a7b-99ec-02319a286260 admin admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:02.861797 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536fb176-feb7-4a7b-99ec-02319a286260 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:02.863769 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536fb176-feb7-4a7b-99ec-02319a286260 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:02.893329 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1096/2210] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:15:02 2022] POST /identity/v3/users => generated 301 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:15:02.898189 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1427b7-1196-43a1-9647-7654fcbe41c0 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:02.910262 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1427b7-1196-43a1-9647-7654fcbe41c0 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:02.917158 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb036a1-ef57-4cd7-84c4-36a85fc4a944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:02.917775 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb036a1-ef57-4cd7-84c4-36a85fc4a944 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:02.918955 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb036a1-ef57-4cd7-84c4-36a85fc4a944 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:02.918955 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb036a1-ef57-4cd7-84c4-36a85fc4a944 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:02.922967 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb036a1-ef57-4cd7-84c4-36a85fc4a944 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:02.922967 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb036a1-ef57-4cd7-84c4-36a85fc4a944 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:02.936279 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1115/2211] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:15:02 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 12:15:02.940163 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536fb176-feb7-4a7b-99ec-02319a286260 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:02.950828 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536fb176-feb7-4a7b-99ec-02319a286260 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:02.956479 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b532f7-c390-4106-acfe-40af0cbd3e74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:02.957241 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b532f7-c390-4106-acfe-40af0cbd3e74 admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:02.957813 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b532f7-c390-4106-acfe-40af0cbd3e74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:02.958614 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b532f7-c390-4106-acfe-40af0cbd3e74 admin admin] PATH_INFO: `/v3/projects/891cb62247e142839d79398e9eff6c20/users/f45fd26ffeab482798c693efcad34a60/roles/3f2ca33902a04b6dbc2ef7d06d7289d3` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:02.975452 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b532f7-c390-4106-acfe-40af0cbd3e74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=891cb62247e142839d79398e9eff6c20, user_id=f45fd26ffeab482798c693efcad34a60, role_id=3f2ca33902a04b6dbc2ef7d06d7289d3)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:02.978352 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b532f7-c390-4106-acfe-40af0cbd3e74 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:02.993233 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1097/2212] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:15:02 2022] PUT /identity/v3/projects/891cb62247e142839d79398e9eff6c20/users/f45fd26ffeab482798c693efcad34a60/roles/3f2ca33902a04b6dbc2ef7d06d7289d3 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:15:02.997989 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb036a1-ef57-4cd7-84c4-36a85fc4a944 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:03.004641 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:15:03.010757 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb036a1-ef57-4cd7-84c4-36a85fc4a944 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:03.099350 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c633d3dc-a785-440b-af6c-335a77653990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:03.101389 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c633d3dc-a785-440b-af6c-335a77653990 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:03.101785 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c633d3dc-a785-440b-af6c-335a77653990 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:03.102140 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c633d3dc-a785-440b-af6c-335a77653990 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:03.103040 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c633d3dc-a785-440b-af6c-335a77653990 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:03.105683 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c633d3dc-a785-440b-af6c-335a77653990 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:03.129072 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1116/2213] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:15:02 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:15:03.133622 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b532f7-c390-4106-acfe-40af0cbd3e74 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:03.146099 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b532f7-c390-4106-acfe-40af0cbd3e74 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:03.152778 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b8c1b7-0771-4e8f-80ca-201625ffd0c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:03.153804 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b8c1b7-0771-4e8f-80ca-201625ffd0c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:03.154096 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b8c1b7-0771-4e8f-80ca-201625ffd0c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:03.154439 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b8c1b7-0771-4e8f-80ca-201625ffd0c5 admin admin] PATH_INFO: `/v3/projects/891cb62247e142839d79398e9eff6c20/users/f45fd26ffeab482798c693efcad34a60/roles/3f2ca33902a04b6dbc2ef7d06d7289d3` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:03.157266 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b8c1b7-0771-4e8f-80ca-201625ffd0c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=891cb62247e142839d79398e9eff6c20, user_id=f45fd26ffeab482798c693efcad34a60, role_id=3f2ca33902a04b6dbc2ef7d06d7289d3)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:03.160680 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b8c1b7-0771-4e8f-80ca-201625ffd0c5 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:03.175488 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1098/2214] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:15:03 2022] PUT /identity/v3/projects/891cb62247e142839d79398e9eff6c20/users/f45fd26ffeab482798c693efcad34a60/roles/3f2ca33902a04b6dbc2ef7d06d7289d3 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:15:03.187327 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053f0c77-5ed6-4264-bff1-bb2985f53b77 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:03.187608 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053f0c77-5ed6-4264-bff1-bb2985f53b77 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:03.187850 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053f0c77-5ed6-4264-bff1-bb2985f53b77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:03.222295 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-053f0c77-5ed6-4264-bff1-bb2985f53b77 None None] MFA Rules not processed for user `f45fd26ffeab482798c693efcad34a60`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:15:03.235735 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-053f0c77-5ed6-4264-bff1-bb2985f53b77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:15:03.414954 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1117/2215] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:15:03 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:15:03.427288 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b8c1b7-0771-4e8f-80ca-201625ffd0c5 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:03.432025 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-85b8c1b7-0771-4e8f-80ca-201625ffd0c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:15:03.496745 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b8c1b7-0771-4e8f-80ca-201625ffd0c5 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:03.500467 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8322a9-6b73-448b-8604-8fc83e162f08 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535c258100df40b58a5ee44b2c21d621', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:03.501096 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8322a9-6b73-448b-8604-8fc83e162f08 service cinder] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:03.501370 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8322a9-6b73-448b-8604-8fc83e162f08 service cinder] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:03.501635 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8322a9-6b73-448b-8604-8fc83e162f08 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:03.510764 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:15:03.513462 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8322a9-6b73-448b-8604-8fc83e162f08 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:03.515308 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8322a9-6b73-448b-8604-8fc83e162f08 service cinder] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:03.528187 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1099/2216] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:15:03 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:15:03.577211 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053f0c77-5ed6-4264-bff1-bb2985f53b77 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:03.578662 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-053f0c77-5ed6-4264-bff1-bb2985f53b77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:15:03.591460 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053f0c77-5ed6-4264-bff1-bb2985f53b77 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:03.595171 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e76c0a-37e3-4f8f-8c37-53d78a77a955 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535c258100df40b58a5ee44b2c21d621', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:03.595785 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e76c0a-37e3-4f8f-8c37-53d78a77a955 service cinder] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:03.596063 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e76c0a-37e3-4f8f-8c37-53d78a77a955 service cinder] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:03.596327 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e76c0a-37e3-4f8f-8c37-53d78a77a955 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:03.654486 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e76c0a-37e3-4f8f-8c37-53d78a77a955 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:03.655962 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e76c0a-37e3-4f8f-8c37-53d78a77a955 service cinder] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:03.664832 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1118/2217] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:15:03 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:15:07.200546 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8322a9-6b73-448b-8604-8fc83e162f08 service cinder] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:07.264684 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8322a9-6b73-448b-8604-8fc83e162f08 service cinder] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:07.269146 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ab45fc-3eb8-4423-b34c-2bd0006ce6ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:07.269774 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ab45fc-3eb8-4423-b34c-2bd0006ce6ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:07.270045 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ab45fc-3eb8-4423-b34c-2bd0006ce6ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:07.270311 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ab45fc-3eb8-4423-b34c-2bd0006ce6ff admin admin] PATH_INFO: `/v3/users/addf4f48d9c843b8b7daca371a842f6c` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:07.271799 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ab45fc-3eb8-4423-b34c-2bd0006ce6ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=addf4f48d9c843b8b7daca371a842f6c)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:07.273952 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ab45fc-3eb8-4423-b34c-2bd0006ce6ff admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:07.322564 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-b2ab45fc-3eb8-4423-b34c-2bd0006ce6ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'addf4f48d9c843b8b7daca371a842f6c'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:07.335634 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-b2ab45fc-3eb8-4423-b34c-2bd0006ce6ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'addf4f48d9c843b8b7daca371a842f6c'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:07.345069 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-b2ab45fc-3eb8-4423-b34c-2bd0006ce6ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'addf4f48d9c843b8b7daca371a842f6c'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:07.354198 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-b2ab45fc-3eb8-4423-b34c-2bd0006ce6ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'addf4f48d9c843b8b7daca371a842f6c'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:07.354915 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-b2ab45fc-3eb8-4423-b34c-2bd0006ce6ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'addf4f48d9c843b8b7daca371a842f6c'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:07.361097 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1100/2218] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 9 12:15:07 2022] DELETE /identity/v3/users/addf4f48d9c843b8b7daca371a842f6c => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:15:07.365708 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4bda0b-cd14-4a91-b8f3-0ef576d1466d None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:07.366196 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4bda0b-cd14-4a91-b8f3-0ef576d1466d None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:07.366308 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4bda0b-cd14-4a91-b8f3-0ef576d1466d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:07.385742 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-4b4bda0b-cd14-4a91-b8f3-0ef576d1466d None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:15:07.387186 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-4b4bda0b-cd14-4a91-b8f3-0ef576d1466d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:15:07.406235 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:15:07.451729 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1119/2219] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:15:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:15:07.462305 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ab45fc-3eb8-4423-b34c-2bd0006ce6ff admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:07.463805 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-b2ab45fc-3eb8-4423-b34c-2bd0006ce6ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:15:07.524544 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ab45fc-3eb8-4423-b34c-2bd0006ce6ff admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:07.528258 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57cff49-8df4-4754-97c6-be4bad435a48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:07.528867 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57cff49-8df4-4754-97c6-be4bad435a48 service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:07.529149 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57cff49-8df4-4754-97c6-be4bad435a48 service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:07.529422 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57cff49-8df4-4754-97c6-be4bad435a48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:07.539545 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57cff49-8df4-4754-97c6-be4bad435a48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:07.541175 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57cff49-8df4-4754-97c6-be4bad435a48 service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:07.551406 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1101/2220] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:15:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:15:07.835295 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4bda0b-cd14-4a91-b8f3-0ef576d1466d None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:07.836729 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-4b4bda0b-cd14-4a91-b8f3-0ef576d1466d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:15:07.850828 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4bda0b-cd14-4a91-b8f3-0ef576d1466d None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:07.855663 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f57fd50-0d00-4cff-817b-0e76ddf99259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:07.856313 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f57fd50-0d00-4cff-817b-0e76ddf99259 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:07.857149 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f57fd50-0d00-4cff-817b-0e76ddf99259 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:07.857442 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f57fd50-0d00-4cff-817b-0e76ddf99259 admin admin] PATH_INFO: `/v3/projects/765a0a2ca4094dc1bcc5fedb201b2b16` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:07.858876 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f57fd50-0d00-4cff-817b-0e76ddf99259 admin admin] RBAC: Authorizing `identity:delete_project(project_id=765a0a2ca4094dc1bcc5fedb201b2b16)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:07.861124 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f57fd50-0d00-4cff-817b-0e76ddf99259 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:07.912316 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-5f57fd50-0d00-4cff-817b-0e76ddf99259 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '765a0a2ca4094dc1bcc5fedb201b2b16', 'request_id': 'req-5f57fd50-0d00-4cff-817b-0e76ddf99259'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:07.918323 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-5f57fd50-0d00-4cff-817b-0e76ddf99259 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '765a0a2ca4094dc1bcc5fedb201b2b16', 'request_id': 'req-5f57fd50-0d00-4cff-817b-0e76ddf99259'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:07.924705 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-5f57fd50-0d00-4cff-817b-0e76ddf99259 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '765a0a2ca4094dc1bcc5fedb201b2b16', 'request_id': 'req-5f57fd50-0d00-4cff-817b-0e76ddf99259'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:07.939448 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-5f57fd50-0d00-4cff-817b-0e76ddf99259 admin admin] The token cache is being invalidate because project 765a0a2ca4094dc1bcc5fedb201b2b16 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81591) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:15:07.939738 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-5f57fd50-0d00-4cff-817b-0e76ddf99259 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:07.940404 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-5f57fd50-0d00-4cff-817b-0e76ddf99259 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:07.942367 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1120/2221] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 9 12:15:07 2022] DELETE /identity/v3/projects/765a0a2ca4094dc1bcc5fedb201b2b16 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:15:07.946570 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57cff49-8df4-4754-97c6-be4bad435a48 service neutron] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:07.956332 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57cff49-8df4-4754-97c6-be4bad435a48 service neutron] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:07.969228 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:15:08.016451 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86de31be-0979-49d8-b618-09ee42e341b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:08.017194 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86de31be-0979-49d8-b618-09ee42e341b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:08.017500 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86de31be-0979-49d8-b618-09ee42e341b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:08.017779 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86de31be-0979-49d8-b618-09ee42e341b7 admin admin] PATH_INFO: `/v3/users/f45fd26ffeab482798c693efcad34a60` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:08.019001 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86de31be-0979-49d8-b618-09ee42e341b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f45fd26ffeab482798c693efcad34a60)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:08.020962 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86de31be-0979-49d8-b618-09ee42e341b7 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:08.077655 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-86de31be-0979-49d8-b618-09ee42e341b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f45fd26ffeab482798c693efcad34a60'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:08.091620 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-86de31be-0979-49d8-b618-09ee42e341b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f45fd26ffeab482798c693efcad34a60'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:08.098578 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-86de31be-0979-49d8-b618-09ee42e341b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f45fd26ffeab482798c693efcad34a60'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:08.107740 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-86de31be-0979-49d8-b618-09ee42e341b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f45fd26ffeab482798c693efcad34a60'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:08.108411 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-86de31be-0979-49d8-b618-09ee42e341b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f45fd26ffeab482798c693efcad34a60'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:08.115499 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1102/2222] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 9 12:15:07 2022] DELETE /identity/v3/users/f45fd26ffeab482798c693efcad34a60 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:15:08.488442 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f57fd50-0d00-4cff-817b-0e76ddf99259 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:08.489884 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-5f57fd50-0d00-4cff-817b-0e76ddf99259 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:15:08.505944 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f57fd50-0d00-4cff-817b-0e76ddf99259 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:08.578250 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804df94c-6dd7-4c4a-964c-a754bb9e2fd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:08.578942 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804df94c-6dd7-4c4a-964c-a754bb9e2fd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:08.579230 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804df94c-6dd7-4c4a-964c-a754bb9e2fd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:08.579501 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804df94c-6dd7-4c4a-964c-a754bb9e2fd5 admin admin] PATH_INFO: `/v3/projects/891cb62247e142839d79398e9eff6c20` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:08.580733 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804df94c-6dd7-4c4a-964c-a754bb9e2fd5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=891cb62247e142839d79398e9eff6c20)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:08.583151 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804df94c-6dd7-4c4a-964c-a754bb9e2fd5 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:08.634930 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-804df94c-6dd7-4c4a-964c-a754bb9e2fd5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '891cb62247e142839d79398e9eff6c20', 'request_id': 'req-804df94c-6dd7-4c4a-964c-a754bb9e2fd5'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:08.636224 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:15:08.642178 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-804df94c-6dd7-4c4a-964c-a754bb9e2fd5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '891cb62247e142839d79398e9eff6c20', 'request_id': 'req-804df94c-6dd7-4c4a-964c-a754bb9e2fd5'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:08.654720 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-804df94c-6dd7-4c4a-964c-a754bb9e2fd5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '891cb62247e142839d79398e9eff6c20', 'request_id': 'req-804df94c-6dd7-4c4a-964c-a754bb9e2fd5'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:08.669971 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-804df94c-6dd7-4c4a-964c-a754bb9e2fd5 admin admin] The token cache is being invalidate because project 891cb62247e142839d79398e9eff6c20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81591) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:15:08.670262 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-804df94c-6dd7-4c4a-964c-a754bb9e2fd5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:08.670888 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-804df94c-6dd7-4c4a-964c-a754bb9e2fd5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:08.672407 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1121/2223] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 9 12:15:08 2022] DELETE /identity/v3/projects/891cb62247e142839d79398e9eff6c20 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:15:08.678543 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aafb5d-73fa-4325-8ce2-c109a66746e8 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:08.679033 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aafb5d-73fa-4325-8ce2-c109a66746e8 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:08.679193 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aafb5d-73fa-4325-8ce2-c109a66746e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:08.699528 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-93aafb5d-73fa-4325-8ce2-c109a66746e8 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:15:08.700737 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-93aafb5d-73fa-4325-8ce2-c109a66746e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:15:08.761848 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1103/2224] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:15:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:15:08.769688 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fa1086-2dba-4847-a608-812bff0cf7dd None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:08.769960 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fa1086-2dba-4847-a608-812bff0cf7dd None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:08.770236 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fa1086-2dba-4847-a608-812bff0cf7dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:08.791302 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-49fa1086-2dba-4847-a608-812bff0cf7dd None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:15:08.792525 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-49fa1086-2dba-4847-a608-812bff0cf7dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:15:08.802258 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1122/2225] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:15:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:15:08.807708 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93aafb5d-73fa-4325-8ce2-c109a66746e8 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:08.817665 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93aafb5d-73fa-4325-8ce2-c109a66746e8 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:08.822584 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526dd1f9-918e-4bc5-b319-a719a9cf5d42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:08.823215 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526dd1f9-918e-4bc5-b319-a719a9cf5d42 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:08.823492 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526dd1f9-918e-4bc5-b319-a719a9cf5d42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:08.823760 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526dd1f9-918e-4bc5-b319-a719a9cf5d42 admin admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:08.824404 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526dd1f9-918e-4bc5-b319-a719a9cf5d42 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:15:08.824705 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526dd1f9-918e-4bc5-b319-a719a9cf5d42 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:08.826332 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526dd1f9-918e-4bc5-b319-a719a9cf5d42 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:08.839148 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1104/2226] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 9 12:15:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:15:08.843539 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fa1086-2dba-4847-a608-812bff0cf7dd None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:08.854043 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fa1086-2dba-4847-a608-812bff0cf7dd None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:08.858641 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530a0242-2f45-4aac-8237-023ac3dbcc36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:08.859270 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530a0242-2f45-4aac-8237-023ac3dbcc36 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:08.859545 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530a0242-2f45-4aac-8237-023ac3dbcc36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:08.859813 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530a0242-2f45-4aac-8237-023ac3dbcc36 admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:08.860467 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530a0242-2f45-4aac-8237-023ac3dbcc36 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:08.862434 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530a0242-2f45-4aac-8237-023ac3dbcc36 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:08.890800 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1123/2227] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 9 12:15:08 2022] POST /identity/v3/projects => generated 371 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:15:08.894764 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526dd1f9-918e-4bc5-b319-a719a9cf5d42 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:08.905999 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526dd1f9-918e-4bc5-b319-a719a9cf5d42 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:08.928058 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:15:08.967803 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051012ee-ab0f-4361-803b-9d1ed784c5d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:08.968424 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051012ee-ab0f-4361-803b-9d1ed784c5d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:08.968699 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051012ee-ab0f-4361-803b-9d1ed784c5d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:08.968966 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051012ee-ab0f-4361-803b-9d1ed784c5d1 admin admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:08.969611 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051012ee-ab0f-4361-803b-9d1ed784c5d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:08.971730 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051012ee-ab0f-4361-803b-9d1ed784c5d1 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:08.995564 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1105/2228] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:15:08 2022] POST /identity/v3/users => generated 286 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:15:08.999483 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530a0242-2f45-4aac-8237-023ac3dbcc36 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:09.010181 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530a0242-2f45-4aac-8237-023ac3dbcc36 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:09.014931 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa0617d-5a1f-431e-b32b-67a39433296c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:09.015487 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa0617d-5a1f-431e-b32b-67a39433296c admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:09.015765 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa0617d-5a1f-431e-b32b-67a39433296c admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:09.016037 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa0617d-5a1f-431e-b32b-67a39433296c admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:09.016680 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa0617d-5a1f-431e-b32b-67a39433296c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:09.018269 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa0617d-5a1f-431e-b32b-67a39433296c admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:09.029090 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1124/2229] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:15:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:15:09.033310 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051012ee-ab0f-4361-803b-9d1ed784c5d1 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:09.045774 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051012ee-ab0f-4361-803b-9d1ed784c5d1 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:09.050369 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f7ecec-97fe-410e-bf08-c942fda467fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:09.051008 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f7ecec-97fe-410e-bf08-c942fda467fc admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:09.051285 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f7ecec-97fe-410e-bf08-c942fda467fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:09.051551 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f7ecec-97fe-410e-bf08-c942fda467fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:09.052192 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f7ecec-97fe-410e-bf08-c942fda467fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:09.054298 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f7ecec-97fe-410e-bf08-c942fda467fc admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:09.067458 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1106/2230] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:15:09 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:15:09.071533 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa0617d-5a1f-431e-b32b-67a39433296c admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:09.081544 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa0617d-5a1f-431e-b32b-67a39433296c admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:09.086894 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3191dd6c-dc44-40af-b480-fbcb2f08f5b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:09.087580 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3191dd6c-dc44-40af-b480-fbcb2f08f5b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:09.087858 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3191dd6c-dc44-40af-b480-fbcb2f08f5b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:09.088126 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3191dd6c-dc44-40af-b480-fbcb2f08f5b1 admin admin] PATH_INFO: `/v3/projects/2e38cedc9117402ba83235d209e1b5c2/users/52139ccd98ad4731a58c2c44804ca4f0/roles/eecd2f160dd844f5906f3d345780c501` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:09.104351 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3191dd6c-dc44-40af-b480-fbcb2f08f5b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e38cedc9117402ba83235d209e1b5c2, user_id=52139ccd98ad4731a58c2c44804ca4f0, role_id=eecd2f160dd844f5906f3d345780c501)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:09.107036 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3191dd6c-dc44-40af-b480-fbcb2f08f5b1 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:09.120229 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1125/2231] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:15:09 2022] PUT /identity/v3/projects/2e38cedc9117402ba83235d209e1b5c2/users/52139ccd98ad4731a58c2c44804ca4f0/roles/eecd2f160dd844f5906f3d345780c501 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:15:09.132153 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ea2a66-2956-42c6-b34e-bf8c293742e0 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:09.132437 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ea2a66-2956-42c6-b34e-bf8c293742e0 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:09.132708 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ea2a66-2956-42c6-b34e-bf8c293742e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:09.163823 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-f1ea2a66-2956-42c6-b34e-bf8c293742e0 None None] MFA Rules not processed for user `52139ccd98ad4731a58c2c44804ca4f0`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:15:09.165048 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-f1ea2a66-2956-42c6-b34e-bf8c293742e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:15:09.248055 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1107/2232] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:15:09 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:15:09.259608 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3191dd6c-dc44-40af-b480-fbcb2f08f5b1 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:09.261282 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-3191dd6c-dc44-40af-b480-fbcb2f08f5b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:15:09.325079 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3191dd6c-dc44-40af-b480-fbcb2f08f5b1 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:09.329031 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549f40e9-187b-49d8-a389-842e648b879c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535c258100df40b58a5ee44b2c21d621', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:09.329836 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549f40e9-187b-49d8-a389-842e648b879c service cinder] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:09.330245 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549f40e9-187b-49d8-a389-842e648b879c service cinder] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:09.330582 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549f40e9-187b-49d8-a389-842e648b879c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:09.340162 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549f40e9-187b-49d8-a389-842e648b879c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:09.341681 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549f40e9-187b-49d8-a389-842e648b879c service cinder] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:09.351938 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1126/2233] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:15:09 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:15:09.377468 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ea2a66-2956-42c6-b34e-bf8c293742e0 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:09.449698 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ea2a66-2956-42c6-b34e-bf8c293742e0 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:09.454730 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b8764d-f988-4610-be9b-e152475ea5ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:09.455503 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b8764d-f988-4610-be9b-e152475ea5ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:09.455784 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b8764d-f988-4610-be9b-e152475ea5ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:09.456060 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b8764d-f988-4610-be9b-e152475ea5ac admin admin] PATH_INFO: `/v3/users/52139ccd98ad4731a58c2c44804ca4f0` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:09.457318 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b8764d-f988-4610-be9b-e152475ea5ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=52139ccd98ad4731a58c2c44804ca4f0)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:09.459903 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b8764d-f988-4610-be9b-e152475ea5ac admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:09.503425 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:15:09.514428 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-99b8764d-f988-4610-be9b-e152475ea5ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52139ccd98ad4731a58c2c44804ca4f0'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:09.527483 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-99b8764d-f988-4610-be9b-e152475ea5ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52139ccd98ad4731a58c2c44804ca4f0'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:09.534447 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-99b8764d-f988-4610-be9b-e152475ea5ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52139ccd98ad4731a58c2c44804ca4f0'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:09.542210 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-99b8764d-f988-4610-be9b-e152475ea5ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52139ccd98ad4731a58c2c44804ca4f0'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:09.542834 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-99b8764d-f988-4610-be9b-e152475ea5ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52139ccd98ad4731a58c2c44804ca4f0'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:09.548796 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1108/2234] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 9 12:15:09 2022] DELETE /identity/v3/users/52139ccd98ad4731a58c2c44804ca4f0 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:15:09.554329 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63647cae-42a0-4819-bd73-a6c757bdb7b3 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:09.554839 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63647cae-42a0-4819-bd73-a6c757bdb7b3 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:09.554964 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63647cae-42a0-4819-bd73-a6c757bdb7b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:09.576002 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-63647cae-42a0-4819-bd73-a6c757bdb7b3 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:15:09.577115 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-63647cae-42a0-4819-bd73-a6c757bdb7b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:15:09.647064 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1127/2235] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:15:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:15:09.656974 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b8764d-f988-4610-be9b-e152475ea5ac admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:09.658506 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-99b8764d-f988-4610-be9b-e152475ea5ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:15:09.719614 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b8764d-f988-4610-be9b-e152475ea5ac admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:09.726367 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590613cc-b066-4d5d-b35e-df830d8d9126 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:09.727026 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590613cc-b066-4d5d-b35e-df830d8d9126 service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:09.727327 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590613cc-b066-4d5d-b35e-df830d8d9126 service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:09.727594 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590613cc-b066-4d5d-b35e-df830d8d9126 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:09.737459 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590613cc-b066-4d5d-b35e-df830d8d9126 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:09.738917 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590613cc-b066-4d5d-b35e-df830d8d9126 service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:09.748352 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1109/2236] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:15:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:15:10.028755 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63647cae-42a0-4819-bd73-a6c757bdb7b3 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:10.030256 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-63647cae-42a0-4819-bd73-a6c757bdb7b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:15:10.043035 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63647cae-42a0-4819-bd73-a6c757bdb7b3 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:10.047725 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642de310-3e78-417f-97b9-b8632e9e3207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:10.048365 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642de310-3e78-417f-97b9-b8632e9e3207 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:10.048647 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642de310-3e78-417f-97b9-b8632e9e3207 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:10.048910 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642de310-3e78-417f-97b9-b8632e9e3207 admin admin] PATH_INFO: `/v3/projects/2e38cedc9117402ba83235d209e1b5c2` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:10.050049 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642de310-3e78-417f-97b9-b8632e9e3207 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e38cedc9117402ba83235d209e1b5c2)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:10.052107 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642de310-3e78-417f-97b9-b8632e9e3207 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:10.070460 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:15:10.109542 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-642de310-3e78-417f-97b9-b8632e9e3207 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e38cedc9117402ba83235d209e1b5c2', 'request_id': 'req-642de310-3e78-417f-97b9-b8632e9e3207'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:10.118697 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-642de310-3e78-417f-97b9-b8632e9e3207 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e38cedc9117402ba83235d209e1b5c2', 'request_id': 'req-642de310-3e78-417f-97b9-b8632e9e3207'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:10.129159 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-642de310-3e78-417f-97b9-b8632e9e3207 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e38cedc9117402ba83235d209e1b5c2', 'request_id': 'req-642de310-3e78-417f-97b9-b8632e9e3207'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:10.142777 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-642de310-3e78-417f-97b9-b8632e9e3207 admin admin] The token cache is being invalidate because project 2e38cedc9117402ba83235d209e1b5c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81591) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:15:10.143113 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-642de310-3e78-417f-97b9-b8632e9e3207 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:10.143741 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-642de310-3e78-417f-97b9-b8632e9e3207 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:10.145287 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1128/2237] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 9 12:15:10 2022] DELETE /identity/v3/projects/2e38cedc9117402ba83235d209e1b5c2 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:15:10.151026 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a63b68d-3cd2-4ffd-84ec-266095322e24 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:10.151336 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a63b68d-3cd2-4ffd-84ec-266095322e24 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:10.151727 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a63b68d-3cd2-4ffd-84ec-266095322e24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:10.172557 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-9a63b68d-3cd2-4ffd-84ec-266095322e24 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:15:10.173754 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-9a63b68d-3cd2-4ffd-84ec-266095322e24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:15:10.232733 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1110/2238] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:15:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:15:10.240616 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ba874e-f64a-46a0-b01b-26a93eb0ecbc None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:10.240917 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ba874e-f64a-46a0-b01b-26a93eb0ecbc None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:10.241193 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ba874e-f64a-46a0-b01b-26a93eb0ecbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:10.262137 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-55ba874e-f64a-46a0-b01b-26a93eb0ecbc None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:15:10.263377 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-55ba874e-f64a-46a0-b01b-26a93eb0ecbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:15:10.271878 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1129/2239] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:15:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:15:10.276408 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a63b68d-3cd2-4ffd-84ec-266095322e24 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:10.286035 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a63b68d-3cd2-4ffd-84ec-266095322e24 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:10.290918 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcb3b44-cf8b-4617-b26c-4d32c121026d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:10.291523 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcb3b44-cf8b-4617-b26c-4d32c121026d admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:10.291793 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcb3b44-cf8b-4617-b26c-4d32c121026d admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:10.292059 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcb3b44-cf8b-4617-b26c-4d32c121026d admin admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:10.292694 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcb3b44-cf8b-4617-b26c-4d32c121026d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:15:10.292987 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcb3b44-cf8b-4617-b26c-4d32c121026d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:10.294581 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcb3b44-cf8b-4617-b26c-4d32c121026d admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:10.306861 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1111/2240] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 9 12:15:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:15:10.313059 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ba874e-f64a-46a0-b01b-26a93eb0ecbc None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:10.322237 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ba874e-f64a-46a0-b01b-26a93eb0ecbc None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:10.326881 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341caec6-d32a-4e4f-8198-9bb9efefb858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:10.327516 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341caec6-d32a-4e4f-8198-9bb9efefb858 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:10.327792 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341caec6-d32a-4e4f-8198-9bb9efefb858 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:10.328056 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341caec6-d32a-4e4f-8198-9bb9efefb858 admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:10.328682 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341caec6-d32a-4e4f-8198-9bb9efefb858 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:10.330567 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341caec6-d32a-4e4f-8198-9bb9efefb858 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:10.358595 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1130/2241] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 9 12:15:10 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:15:10.362690 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcb3b44-cf8b-4617-b26c-4d32c121026d admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:10.373912 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcb3b44-cf8b-4617-b26c-4d32c121026d admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:10.415885 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:15:10.434953 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fae9bdd-7d1a-4f89-8fce-e65c394d5c90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:10.435817 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fae9bdd-7d1a-4f89-8fce-e65c394d5c90 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:10.436379 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fae9bdd-7d1a-4f89-8fce-e65c394d5c90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:10.436809 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fae9bdd-7d1a-4f89-8fce-e65c394d5c90 admin admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:10.437636 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fae9bdd-7d1a-4f89-8fce-e65c394d5c90 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:10.440132 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fae9bdd-7d1a-4f89-8fce-e65c394d5c90 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:10.461916 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1112/2242] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:15:10 2022] POST /identity/v3/users => generated 292 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:15:10.466318 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341caec6-d32a-4e4f-8198-9bb9efefb858 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:10.476757 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341caec6-d32a-4e4f-8198-9bb9efefb858 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:10.481378 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9bd484-71da-4abf-975b-f0cc2d956eb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:10.481976 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9bd484-71da-4abf-975b-f0cc2d956eb0 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:10.482256 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9bd484-71da-4abf-975b-f0cc2d956eb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:10.482532 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9bd484-71da-4abf-975b-f0cc2d956eb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:10.483221 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9bd484-71da-4abf-975b-f0cc2d956eb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:10.484815 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9bd484-71da-4abf-975b-f0cc2d956eb0 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:10.495120 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1131/2243] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:15:10 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:15:10.498846 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fae9bdd-7d1a-4f89-8fce-e65c394d5c90 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:10.510057 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fae9bdd-7d1a-4f89-8fce-e65c394d5c90 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:10.514718 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd02bf18-0299-43b3-9c5a-a8bdfb80189e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:10.515350 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd02bf18-0299-43b3-9c5a-a8bdfb80189e admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:10.515626 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd02bf18-0299-43b3-9c5a-a8bdfb80189e admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:10.515965 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd02bf18-0299-43b3-9c5a-a8bdfb80189e admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:10.516528 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd02bf18-0299-43b3-9c5a-a8bdfb80189e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:10.518117 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd02bf18-0299-43b3-9c5a-a8bdfb80189e admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:10.537749 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1113/2244] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:15:10 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:15:10.541888 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9bd484-71da-4abf-975b-f0cc2d956eb0 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:10.553718 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9bd484-71da-4abf-975b-f0cc2d956eb0 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:10.562795 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0047fb-9672-493c-80ca-fee6837a72b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:10.563557 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0047fb-9672-493c-80ca-fee6837a72b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:10.563685 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0047fb-9672-493c-80ca-fee6837a72b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:10.564011 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0047fb-9672-493c-80ca-fee6837a72b1 admin admin] PATH_INFO: `/v3/projects/552cf622ee984d55bad9215de80cae43/users/9dde25daae0e49baab329bb668868d21/roles/eecd2f160dd844f5906f3d345780c501` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:10.581846 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0047fb-9672-493c-80ca-fee6837a72b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=552cf622ee984d55bad9215de80cae43, user_id=9dde25daae0e49baab329bb668868d21, role_id=eecd2f160dd844f5906f3d345780c501)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:10.584958 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0047fb-9672-493c-80ca-fee6837a72b1 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:10.606726 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1132/2245] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:15:10 2022] PUT /identity/v3/projects/552cf622ee984d55bad9215de80cae43/users/9dde25daae0e49baab329bb668868d21/roles/eecd2f160dd844f5906f3d345780c501 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:15:10.618069 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657e3baa-ff30-45da-b0e4-1b2bd6bef1d8 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:10.618347 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657e3baa-ff30-45da-b0e4-1b2bd6bef1d8 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:10.618622 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657e3baa-ff30-45da-b0e4-1b2bd6bef1d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:10.653129 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-657e3baa-ff30-45da-b0e4-1b2bd6bef1d8 None None] MFA Rules not processed for user `9dde25daae0e49baab329bb668868d21`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:15:10.654546 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-657e3baa-ff30-45da-b0e4-1b2bd6bef1d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:15:10.734742 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1114/2246] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:15:10 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:15:10.739218 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0047fb-9672-493c-80ca-fee6837a72b1 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:10.807564 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0047fb-9672-493c-80ca-fee6837a72b1 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:10.813249 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7597ae8-9f28-4409-af2e-01fafbef0eba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:10.813857 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7597ae8-9f28-4409-af2e-01fafbef0eba admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:10.814136 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7597ae8-9f28-4409-af2e-01fafbef0eba admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:10.814414 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7597ae8-9f28-4409-af2e-01fafbef0eba admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:10.815219 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7597ae8-9f28-4409-af2e-01fafbef0eba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:10.817305 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7597ae8-9f28-4409-af2e-01fafbef0eba admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:10.848344 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1133/2247] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 9 12:15:10 2022] POST /identity/v3/projects => generated 383 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:15:10.854784 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657e3baa-ff30-45da-b0e4-1b2bd6bef1d8 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:10.871577 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657e3baa-ff30-45da-b0e4-1b2bd6bef1d8 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:10.959886 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372a742e-3f6b-425b-88bb-62d1e31c9ab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:10.960620 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372a742e-3f6b-425b-88bb-62d1e31c9ab5 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:10.961016 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372a742e-3f6b-425b-88bb-62d1e31c9ab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:10.961191 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372a742e-3f6b-425b-88bb-62d1e31c9ab5 admin admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:10.961793 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372a742e-3f6b-425b-88bb-62d1e31c9ab5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:10.964147 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372a742e-3f6b-425b-88bb-62d1e31c9ab5 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:11.002325 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1115/2248] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:15:10 2022] POST /identity/v3/users => generated 292 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:15:11.007477 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7597ae8-9f28-4409-af2e-01fafbef0eba admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:11.022224 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7597ae8-9f28-4409-af2e-01fafbef0eba admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:11.027672 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096f92d2-5e92-404a-aa8f-df69abd53363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:11.028283 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096f92d2-5e92-404a-aa8f-df69abd53363 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:11.028616 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096f92d2-5e92-404a-aa8f-df69abd53363 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:11.028892 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096f92d2-5e92-404a-aa8f-df69abd53363 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:11.029546 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096f92d2-5e92-404a-aa8f-df69abd53363 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:11.031195 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096f92d2-5e92-404a-aa8f-df69abd53363 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:11.046316 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1134/2249] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:15:11 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:15:11.047673 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372a742e-3f6b-425b-88bb-62d1e31c9ab5 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:11.059149 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372a742e-3f6b-425b-88bb-62d1e31c9ab5 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:11.063773 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34acfbf7-908e-4381-90d1-e132500dd276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:11.064365 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34acfbf7-908e-4381-90d1-e132500dd276 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:11.064643 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34acfbf7-908e-4381-90d1-e132500dd276 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:11.064912 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34acfbf7-908e-4381-90d1-e132500dd276 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:11.065542 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34acfbf7-908e-4381-90d1-e132500dd276 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:11.067172 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34acfbf7-908e-4381-90d1-e132500dd276 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:11.079973 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1116/2250] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:15:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:15:11.081319 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096f92d2-5e92-404a-aa8f-df69abd53363 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:11.093250 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096f92d2-5e92-404a-aa8f-df69abd53363 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:11.098299 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506ce460-ceb1-4898-ae81-9b801297f5e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:11.099013 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506ce460-ceb1-4898-ae81-9b801297f5e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:11.100020 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506ce460-ceb1-4898-ae81-9b801297f5e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:11.100508 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506ce460-ceb1-4898-ae81-9b801297f5e9 admin admin] PATH_INFO: `/v3/projects/32d87541a1df4d4a86ae09de3affe436/users/d89598a36d8948299df468df713657c9/roles/eecd2f160dd844f5906f3d345780c501` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:11.108726 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:15:11.117457 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506ce460-ceb1-4898-ae81-9b801297f5e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=32d87541a1df4d4a86ae09de3affe436, user_id=d89598a36d8948299df468df713657c9, role_id=eecd2f160dd844f5906f3d345780c501)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:11.120599 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506ce460-ceb1-4898-ae81-9b801297f5e9 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:11.133731 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1135/2251] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:15:11 2022] PUT /identity/v3/projects/32d87541a1df4d4a86ae09de3affe436/users/d89598a36d8948299df468df713657c9/roles/eecd2f160dd844f5906f3d345780c501 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:15:11.206294 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78a3368-a337-446d-92bf-c45b61c0bb39 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:11.206832 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78a3368-a337-446d-92bf-c45b61c0bb39 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:11.207139 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78a3368-a337-446d-92bf-c45b61c0bb39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:11.239229 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-c78a3368-a337-446d-92bf-c45b61c0bb39 None None] MFA Rules not processed for user `d89598a36d8948299df468df713657c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:15:11.241085 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-c78a3368-a337-446d-92bf-c45b61c0bb39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:15:11.259835 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:15:11.312089 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1117/2252] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:15:11 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:15:11.316905 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506ce460-ceb1-4898-ae81-9b801297f5e9 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:11.385288 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506ce460-ceb1-4898-ae81-9b801297f5e9 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:11.391970 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af79692f-9938-4c25-8ea7-dda71c3e2c6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:11.392825 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af79692f-9938-4c25-8ea7-dda71c3e2c6b admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:11.393111 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af79692f-9938-4c25-8ea7-dda71c3e2c6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:11.393384 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af79692f-9938-4c25-8ea7-dda71c3e2c6b admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:11.394029 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af79692f-9938-4c25-8ea7-dda71c3e2c6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:11.396074 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af79692f-9938-4c25-8ea7-dda71c3e2c6b admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:11.426642 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1136/2253] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 9 12:15:11 2022] POST /identity/v3/projects => generated 385 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:15:11.431481 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78a3368-a337-446d-92bf-c45b61c0bb39 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:11.443634 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78a3368-a337-446d-92bf-c45b61c0bb39 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:11.504599 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7381a87-3aa9-47c0-bcd5-d9b23db3b176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:11.505269 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7381a87-3aa9-47c0-bcd5-d9b23db3b176 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:11.505600 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7381a87-3aa9-47c0-bcd5-d9b23db3b176 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:11.505882 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7381a87-3aa9-47c0-bcd5-d9b23db3b176 admin admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:11.506515 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7381a87-3aa9-47c0-bcd5-d9b23db3b176 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:11.508513 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7381a87-3aa9-47c0-bcd5-d9b23db3b176 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:11.531876 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1118/2254] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:15:11 2022] POST /identity/v3/users => generated 299 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:15:11.535961 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af79692f-9938-4c25-8ea7-dda71c3e2c6b admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:11.547527 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af79692f-9938-4c25-8ea7-dda71c3e2c6b admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:11.554192 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d954bb32-3587-4453-b526-f9b8e76a36f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:11.554798 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d954bb32-3587-4453-b526-f9b8e76a36f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:11.555097 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d954bb32-3587-4453-b526-f9b8e76a36f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:11.555368 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d954bb32-3587-4453-b526-f9b8e76a36f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:11.556039 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d954bb32-3587-4453-b526-f9b8e76a36f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:11.557661 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d954bb32-3587-4453-b526-f9b8e76a36f1 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:11.570025 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1137/2255] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:15:11 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:15:11.574301 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7381a87-3aa9-47c0-bcd5-d9b23db3b176 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:11.584713 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7381a87-3aa9-47c0-bcd5-d9b23db3b176 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:11.590297 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438b8c44-93f0-4947-8893-608d9aff8314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:11.591076 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438b8c44-93f0-4947-8893-608d9aff8314 admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:11.591357 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438b8c44-93f0-4947-8893-608d9aff8314 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:11.591744 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438b8c44-93f0-4947-8893-608d9aff8314 admin admin] PATH_INFO: `/v3/projects/d41e574a9a94497fa3ea67f4095afcec/users/2ffad99186e7403c98751e2d679a73a6/roles/3f2ca33902a04b6dbc2ef7d06d7289d3` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:11.607884 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438b8c44-93f0-4947-8893-608d9aff8314 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d41e574a9a94497fa3ea67f4095afcec, user_id=2ffad99186e7403c98751e2d679a73a6, role_id=3f2ca33902a04b6dbc2ef7d06d7289d3)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:11.610616 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438b8c44-93f0-4947-8893-608d9aff8314 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:11.625379 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1119/2256] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:15:11 2022] PUT /identity/v3/projects/d41e574a9a94497fa3ea67f4095afcec/users/2ffad99186e7403c98751e2d679a73a6/roles/3f2ca33902a04b6dbc2ef7d06d7289d3 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:15:11.629353 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d954bb32-3587-4453-b526-f9b8e76a36f1 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:11.639585 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d954bb32-3587-4453-b526-f9b8e76a36f1 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:11.698165 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823c25dd-2184-4e2d-9cdb-f13f2f1ab3a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:11.698775 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823c25dd-2184-4e2d-9cdb-f13f2f1ab3a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:11.699085 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823c25dd-2184-4e2d-9cdb-f13f2f1ab3a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:11.699358 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823c25dd-2184-4e2d-9cdb-f13f2f1ab3a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:11.699996 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823c25dd-2184-4e2d-9cdb-f13f2f1ab3a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:11.701597 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823c25dd-2184-4e2d-9cdb-f13f2f1ab3a9 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:11.711441 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1138/2257] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:15:11 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:15:11.715559 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438b8c44-93f0-4947-8893-608d9aff8314 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:11.728376 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438b8c44-93f0-4947-8893-608d9aff8314 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:11.732980 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58381d12-06a3-4f81-a42a-449077f87e5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:11.733634 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58381d12-06a3-4f81-a42a-449077f87e5b admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:11.734031 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58381d12-06a3-4f81-a42a-449077f87e5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:11.734302 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58381d12-06a3-4f81-a42a-449077f87e5b admin admin] PATH_INFO: `/v3/projects/d41e574a9a94497fa3ea67f4095afcec/users/2ffad99186e7403c98751e2d679a73a6/roles/3f2ca33902a04b6dbc2ef7d06d7289d3` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:11.736222 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58381d12-06a3-4f81-a42a-449077f87e5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d41e574a9a94497fa3ea67f4095afcec, user_id=2ffad99186e7403c98751e2d679a73a6, role_id=3f2ca33902a04b6dbc2ef7d06d7289d3)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:11.740456 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58381d12-06a3-4f81-a42a-449077f87e5b admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:11.752340 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1120/2258] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:15:11 2022] PUT /identity/v3/projects/d41e574a9a94497fa3ea67f4095afcec/users/2ffad99186e7403c98751e2d679a73a6/roles/3f2ca33902a04b6dbc2ef7d06d7289d3 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:15:11.762533 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a8cac1-f8fb-4ebd-9b0e-c0aa34d9eac3 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:11.762820 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a8cac1-f8fb-4ebd-9b0e-c0aa34d9eac3 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:11.763132 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a8cac1-f8fb-4ebd-9b0e-c0aa34d9eac3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:11.794436 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-c4a8cac1-f8fb-4ebd-9b0e-c0aa34d9eac3 None None] MFA Rules not processed for user `2ffad99186e7403c98751e2d679a73a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:15:11.796182 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-c4a8cac1-f8fb-4ebd-9b0e-c0aa34d9eac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:15:11.843790 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:15:11.872204 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1139/2259] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:15:11 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:15:11.888539 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58381d12-06a3-4f81-a42a-449077f87e5b admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:11.890336 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-58381d12-06a3-4f81-a42a-449077f87e5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:15:11.952319 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58381d12-06a3-4f81-a42a-449077f87e5b admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:11.956035 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100e4aa0-2471-442a-8e1b-6fc8a2226efe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535c258100df40b58a5ee44b2c21d621', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:11.956641 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100e4aa0-2471-442a-8e1b-6fc8a2226efe service cinder] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:11.956919 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100e4aa0-2471-442a-8e1b-6fc8a2226efe service cinder] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:11.957190 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100e4aa0-2471-442a-8e1b-6fc8a2226efe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:11.986662 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:15:12.023691 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100e4aa0-2471-442a-8e1b-6fc8a2226efe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:12.025180 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100e4aa0-2471-442a-8e1b-6fc8a2226efe service cinder] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:12.034808 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1121/2260] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:15:11 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:15:13.739447 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a8cac1-f8fb-4ebd-9b0e-c0aa34d9eac3 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:13.749846 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a8cac1-f8fb-4ebd-9b0e-c0aa34d9eac3 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:13.753678 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985caaf1-c3cf-4c1b-b2d1-778c46fa3a24 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535c258100df40b58a5ee44b2c21d621', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:13.754318 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985caaf1-c3cf-4c1b-b2d1-778c46fa3a24 service cinder] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:13.754595 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985caaf1-c3cf-4c1b-b2d1-778c46fa3a24 service cinder] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:13.754865 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985caaf1-c3cf-4c1b-b2d1-778c46fa3a24 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:13.820783 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985caaf1-c3cf-4c1b-b2d1-778c46fa3a24 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:13.822448 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985caaf1-c3cf-4c1b-b2d1-778c46fa3a24 service cinder] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:13.839970 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1140/2261] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:15:13 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:15:13.902671 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0522f599-4cd3-4065-8d06-9fbc25804ebf None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:13.903183 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0522f599-4cd3-4065-8d06-9fbc25804ebf None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:13.905412 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0522f599-4cd3-4065-8d06-9fbc25804ebf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:13.933039 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-0522f599-4cd3-4065-8d06-9fbc25804ebf None None] MFA Rules not processed for user `2e125f66d9c84e9797099065351a07da`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:15:13.934799 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-0522f599-4cd3-4065-8d06-9fbc25804ebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:15:14.010045 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1122/2262] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:15:13 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:15:14.545228 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985caaf1-c3cf-4c1b-b2d1-778c46fa3a24 service cinder] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:14.546677 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-985caaf1-c3cf-4c1b-b2d1-778c46fa3a24 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:15:14.559666 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985caaf1-c3cf-4c1b-b2d1-778c46fa3a24 service cinder] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:14.563403 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3288b0b-4de5-4a0c-a26e-4ad23b49792a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535c258100df40b58a5ee44b2c21d621', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:14.564118 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3288b0b-4de5-4a0c-a26e-4ad23b49792a service cinder] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:14.564382 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3288b0b-4de5-4a0c-a26e-4ad23b49792a service cinder] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:14.564655 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3288b0b-4de5-4a0c-a26e-4ad23b49792a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:14.575627 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3288b0b-4de5-4a0c-a26e-4ad23b49792a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:14.577088 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3288b0b-4de5-4a0c-a26e-4ad23b49792a service cinder] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:14.588730 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1141/2263] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:15:14 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:15:18.735842 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8a5ae9-d161-418f-a1df-3190cb3f7eae None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:18.736352 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8a5ae9-d161-418f-a1df-3190cb3f7eae None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:18.736445 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8a5ae9-d161-418f-a1df-3190cb3f7eae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:18.757070 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-af8a5ae9-d161-418f-a1df-3190cb3f7eae None None] MFA Rules not processed for user `e368b4e668c54e25bff80a8e8a8c3588`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:15:18.758569 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-af8a5ae9-d161-418f-a1df-3190cb3f7eae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:15:18.815571 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1123/2264] 10.222.0.49 () {58 vars in 1009 bytes} [Fri Sep 9 12:15:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:15:18.861132 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3288b0b-4de5-4a0c-a26e-4ad23b49792a service cinder] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:18.862565 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-e3288b0b-4de5-4a0c-a26e-4ad23b49792a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:15:18.949836 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3288b0b-4de5-4a0c-a26e-4ad23b49792a service cinder] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:18.957738 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa34478a-6d10-4e42-9033-67f261dac1b2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e652ba827849b5a3fc1213d04a1b76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:18.958357 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa34478a-6d10-4e42-9033-67f261dac1b2 service swift] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:18.958636 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa34478a-6d10-4e42-9033-67f261dac1b2 service swift] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:18.958922 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa34478a-6d10-4e42-9033-67f261dac1b2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:18.968802 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa34478a-6d10-4e42-9033-67f261dac1b2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:18.970375 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa34478a-6d10-4e42-9033-67f261dac1b2 service swift] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:18.980319 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1142/2265] 10.222.0.49 () {60 vars in 1384 bytes} [Fri Sep 9 12:15:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 12:15:22.009126 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8a5ae9-d161-418f-a1df-3190cb3f7eae None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:22.122743 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8a5ae9-d161-418f-a1df-3190cb3f7eae None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:22.129860 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35510e28-6f54-4df6-ab64-0577aa23b213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:22.130917 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35510e28-6f54-4df6-ab64-0577aa23b213 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:22.132457 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35510e28-6f54-4df6-ab64-0577aa23b213 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:22.132733 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35510e28-6f54-4df6-ab64-0577aa23b213 admin admin] PATH_INFO: `/v3/users/9dde25daae0e49baab329bb668868d21` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:22.134223 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35510e28-6f54-4df6-ab64-0577aa23b213 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9dde25daae0e49baab329bb668868d21)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:22.136683 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35510e28-6f54-4df6-ab64-0577aa23b213 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:22.167685 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:15:22.220559 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-35510e28-6f54-4df6-ab64-0577aa23b213 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9dde25daae0e49baab329bb668868d21'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:22.243724 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-35510e28-6f54-4df6-ab64-0577aa23b213 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9dde25daae0e49baab329bb668868d21'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:22.258023 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-35510e28-6f54-4df6-ab64-0577aa23b213 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9dde25daae0e49baab329bb668868d21'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:22.276324 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-35510e28-6f54-4df6-ab64-0577aa23b213 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9dde25daae0e49baab329bb668868d21'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:22.277072 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-35510e28-6f54-4df6-ab64-0577aa23b213 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9dde25daae0e49baab329bb668868d21'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:22.291171 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1124/2266] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 9 12:15:22 2022] DELETE /identity/v3/users/9dde25daae0e49baab329bb668868d21 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:15:22.292223 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ef2943-9a79-463f-b4b4-3f1883dfb5d6 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:22.292731 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ef2943-9a79-463f-b4b4-3f1883dfb5d6 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:22.292833 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ef2943-9a79-463f-b4b4-3f1883dfb5d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:22.323994 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-66ef2943-9a79-463f-b4b4-3f1883dfb5d6 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:15:22.326718 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-66ef2943-9a79-463f-b4b4-3f1883dfb5d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:15:22.439415 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:15:22.461359 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1143/2267] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:15:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:15:22.473963 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35510e28-6f54-4df6-ab64-0577aa23b213 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:22.476376 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-35510e28-6f54-4df6-ab64-0577aa23b213 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:15:22.585976 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35510e28-6f54-4df6-ab64-0577aa23b213 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:22.602532 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3ce7ae-876a-40e6-805d-b8f05a4788a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:22.603203 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3ce7ae-876a-40e6-805d-b8f05a4788a0 service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:22.603481 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3ce7ae-876a-40e6-805d-b8f05a4788a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:22.603752 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3ce7ae-876a-40e6-805d-b8f05a4788a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:22.616082 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3ce7ae-876a-40e6-805d-b8f05a4788a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:22.617902 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3ce7ae-876a-40e6-805d-b8f05a4788a0 service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:22.632769 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1125/2268] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:15:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:15:23.219027 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ef2943-9a79-463f-b4b4-3f1883dfb5d6 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:23.220582 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-66ef2943-9a79-463f-b4b4-3f1883dfb5d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:15:23.235614 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ef2943-9a79-463f-b4b4-3f1883dfb5d6 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:23.246791 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8032e9bf-c62e-4527-a996-e97b67f3675b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:23.247797 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8032e9bf-c62e-4527-a996-e97b67f3675b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:23.248306 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8032e9bf-c62e-4527-a996-e97b67f3675b admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:23.248786 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8032e9bf-c62e-4527-a996-e97b67f3675b admin admin] PATH_INFO: `/v3/projects/552cf622ee984d55bad9215de80cae43` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:23.250312 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8032e9bf-c62e-4527-a996-e97b67f3675b admin admin] RBAC: Authorizing `identity:delete_project(project_id=552cf622ee984d55bad9215de80cae43)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:23.252638 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8032e9bf-c62e-4527-a996-e97b67f3675b admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:23.304424 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-8032e9bf-c62e-4527-a996-e97b67f3675b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '552cf622ee984d55bad9215de80cae43', 'request_id': 'req-8032e9bf-c62e-4527-a996-e97b67f3675b'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:23.310101 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-8032e9bf-c62e-4527-a996-e97b67f3675b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '552cf622ee984d55bad9215de80cae43', 'request_id': 'req-8032e9bf-c62e-4527-a996-e97b67f3675b'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:23.316290 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-8032e9bf-c62e-4527-a996-e97b67f3675b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '552cf622ee984d55bad9215de80cae43', 'request_id': 'req-8032e9bf-c62e-4527-a996-e97b67f3675b'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:23.328371 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-8032e9bf-c62e-4527-a996-e97b67f3675b admin admin] The token cache is being invalidate because project 552cf622ee984d55bad9215de80cae43 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81591) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:15:23.328657 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-8032e9bf-c62e-4527-a996-e97b67f3675b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:23.329271 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-8032e9bf-c62e-4527-a996-e97b67f3675b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:23.330756 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1144/2269] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 9 12:15:23 2022] DELETE /identity/v3/projects/552cf622ee984d55bad9215de80cae43 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:15:23.335811 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3ce7ae-876a-40e6-805d-b8f05a4788a0 service neutron] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:23.346569 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3ce7ae-876a-40e6-805d-b8f05a4788a0 service neutron] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:23.416483 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793d6d6b-f3b5-460b-ac06-d100ee410db0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:23.417227 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793d6d6b-f3b5-460b-ac06-d100ee410db0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:23.417611 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793d6d6b-f3b5-460b-ac06-d100ee410db0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:23.417774 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793d6d6b-f3b5-460b-ac06-d100ee410db0 admin admin] PATH_INFO: `/v3/users/d89598a36d8948299df468df713657c9` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:23.419021 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793d6d6b-f3b5-460b-ac06-d100ee410db0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d89598a36d8948299df468df713657c9)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:23.420970 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793d6d6b-f3b5-460b-ac06-d100ee410db0 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:23.467337 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:15:23.476083 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-793d6d6b-f3b5-460b-ac06-d100ee410db0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd89598a36d8948299df468df713657c9'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:23.489217 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-793d6d6b-f3b5-460b-ac06-d100ee410db0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd89598a36d8948299df468df713657c9'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:23.496986 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-793d6d6b-f3b5-460b-ac06-d100ee410db0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd89598a36d8948299df468df713657c9'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:23.504019 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-793d6d6b-f3b5-460b-ac06-d100ee410db0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd89598a36d8948299df468df713657c9'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:23.504653 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-793d6d6b-f3b5-460b-ac06-d100ee410db0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd89598a36d8948299df468df713657c9'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:23.511500 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1126/2270] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 9 12:15:23 2022] DELETE /identity/v3/users/d89598a36d8948299df468df713657c9 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:15:23.889943 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8032e9bf-c62e-4527-a996-e97b67f3675b admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:23.891625 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-8032e9bf-c62e-4527-a996-e97b67f3675b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:15:23.928010 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8032e9bf-c62e-4527-a996-e97b67f3675b admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:24.001970 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3c2ec2-a6a2-4672-abbe-84238140b02e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:24.002670 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3c2ec2-a6a2-4672-abbe-84238140b02e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:24.002989 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3c2ec2-a6a2-4672-abbe-84238140b02e admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:24.003388 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3c2ec2-a6a2-4672-abbe-84238140b02e admin admin] PATH_INFO: `/v3/projects/32d87541a1df4d4a86ae09de3affe436` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:24.004523 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3c2ec2-a6a2-4672-abbe-84238140b02e admin admin] RBAC: Authorizing `identity:delete_project(project_id=32d87541a1df4d4a86ae09de3affe436)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:24.006849 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3c2ec2-a6a2-4672-abbe-84238140b02e admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:24.069803 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-2f3c2ec2-a6a2-4672-abbe-84238140b02e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32d87541a1df4d4a86ae09de3affe436', 'request_id': 'req-2f3c2ec2-a6a2-4672-abbe-84238140b02e'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:24.076797 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-2f3c2ec2-a6a2-4672-abbe-84238140b02e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32d87541a1df4d4a86ae09de3affe436', 'request_id': 'req-2f3c2ec2-a6a2-4672-abbe-84238140b02e'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:24.088375 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-2f3c2ec2-a6a2-4672-abbe-84238140b02e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32d87541a1df4d4a86ae09de3affe436', 'request_id': 'req-2f3c2ec2-a6a2-4672-abbe-84238140b02e'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:24.104894 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-2f3c2ec2-a6a2-4672-abbe-84238140b02e admin admin] The token cache is being invalidate because project 32d87541a1df4d4a86ae09de3affe436 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81591) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:15:24.105260 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-2f3c2ec2-a6a2-4672-abbe-84238140b02e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:24.105898 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-2f3c2ec2-a6a2-4672-abbe-84238140b02e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:24.107685 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1145/2271] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 9 12:15:23 2022] DELETE /identity/v3/projects/32d87541a1df4d4a86ae09de3affe436 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:15:24.113001 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793d6d6b-f3b5-460b-ac06-d100ee410db0 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:24.124284 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-793d6d6b-f3b5-460b-ac06-d100ee410db0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:15:24.168323 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793d6d6b-f3b5-460b-ac06-d100ee410db0 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:24.291506 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487c78a3-9111-4d85-8298-195db4b54c2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:24.292182 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487c78a3-9111-4d85-8298-195db4b54c2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:24.292596 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487c78a3-9111-4d85-8298-195db4b54c2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:24.293549 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487c78a3-9111-4d85-8298-195db4b54c2a admin admin] PATH_INFO: `/v3/users/2ffad99186e7403c98751e2d679a73a6` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:24.296623 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487c78a3-9111-4d85-8298-195db4b54c2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ffad99186e7403c98751e2d679a73a6)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:24.311442 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487c78a3-9111-4d85-8298-195db4b54c2a admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:24.451065 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-487c78a3-9111-4d85-8298-195db4b54c2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ffad99186e7403c98751e2d679a73a6'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:24.463558 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-487c78a3-9111-4d85-8298-195db4b54c2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ffad99186e7403c98751e2d679a73a6'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:24.479536 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-487c78a3-9111-4d85-8298-195db4b54c2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ffad99186e7403c98751e2d679a73a6'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:24.516617 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-487c78a3-9111-4d85-8298-195db4b54c2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ffad99186e7403c98751e2d679a73a6'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:24.517751 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-487c78a3-9111-4d85-8298-195db4b54c2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ffad99186e7403c98751e2d679a73a6'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:24.540976 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1127/2272] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 9 12:15:24 2022] DELETE /identity/v3/users/2ffad99186e7403c98751e2d679a73a6 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:15:25.113017 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3c2ec2-a6a2-4672-abbe-84238140b02e admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:25.115006 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-2f3c2ec2-a6a2-4672-abbe-84238140b02e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:15:25.159678 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3c2ec2-a6a2-4672-abbe-84238140b02e admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:25.170278 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:15:25.223362 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116cbf95-7b7b-4adb-9e1f-d3e8facf1941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:25.223997 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116cbf95-7b7b-4adb-9e1f-d3e8facf1941 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:25.224273 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116cbf95-7b7b-4adb-9e1f-d3e8facf1941 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:25.224543 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116cbf95-7b7b-4adb-9e1f-d3e8facf1941 admin admin] PATH_INFO: `/v3/projects/d41e574a9a94497fa3ea67f4095afcec` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:25.226754 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116cbf95-7b7b-4adb-9e1f-d3e8facf1941 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d41e574a9a94497fa3ea67f4095afcec)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:25.228703 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116cbf95-7b7b-4adb-9e1f-d3e8facf1941 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:25.277255 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-116cbf95-7b7b-4adb-9e1f-d3e8facf1941 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd41e574a9a94497fa3ea67f4095afcec', 'request_id': 'req-116cbf95-7b7b-4adb-9e1f-d3e8facf1941'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:25.282633 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-116cbf95-7b7b-4adb-9e1f-d3e8facf1941 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd41e574a9a94497fa3ea67f4095afcec', 'request_id': 'req-116cbf95-7b7b-4adb-9e1f-d3e8facf1941'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:25.289607 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-116cbf95-7b7b-4adb-9e1f-d3e8facf1941 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd41e574a9a94497fa3ea67f4095afcec', 'request_id': 'req-116cbf95-7b7b-4adb-9e1f-d3e8facf1941'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:25.303635 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-116cbf95-7b7b-4adb-9e1f-d3e8facf1941 admin admin] The token cache is being invalidate because project d41e574a9a94497fa3ea67f4095afcec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81591) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:15:25.303921 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-116cbf95-7b7b-4adb-9e1f-d3e8facf1941 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:25.304567 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-116cbf95-7b7b-4adb-9e1f-d3e8facf1941 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:25.306345 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1146/2273] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 9 12:15:25 2022] DELETE /identity/v3/projects/d41e574a9a94497fa3ea67f4095afcec => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:15:25.311326 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c564b6b-8dc8-469e-80e8-0f3755d5436c None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:25.311729 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c564b6b-8dc8-469e-80e8-0f3755d5436c None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:25.312220 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c564b6b-8dc8-469e-80e8-0f3755d5436c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:25.336299 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-3c564b6b-8dc8-469e-80e8-0f3755d5436c None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:15:25.337752 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-3c564b6b-8dc8-469e-80e8-0f3755d5436c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:15:25.383125 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:15:25.454749 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1128/2274] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:15:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:15:25.464605 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44550522-a827-49ec-b6d5-9ede5c1468c1 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:25.464897 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44550522-a827-49ec-b6d5-9ede5c1468c1 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:25.465168 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44550522-a827-49ec-b6d5-9ede5c1468c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:25.489086 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-44550522-a827-49ec-b6d5-9ede5c1468c1 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:15:25.491166 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-44550522-a827-49ec-b6d5-9ede5c1468c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:15:25.500770 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1147/2275] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:15:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:15:25.505038 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c564b6b-8dc8-469e-80e8-0f3755d5436c None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:25.522282 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c564b6b-8dc8-469e-80e8-0f3755d5436c None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:25.531103 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e3f87b-4fb8-428e-a7d0-d343d38ae16e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:25.531729 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e3f87b-4fb8-428e-a7d0-d343d38ae16e admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:25.532009 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e3f87b-4fb8-428e-a7d0-d343d38ae16e admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:25.532281 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e3f87b-4fb8-428e-a7d0-d343d38ae16e admin admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:25.533682 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e3f87b-4fb8-428e-a7d0-d343d38ae16e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:15:25.534001 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e3f87b-4fb8-428e-a7d0-d343d38ae16e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:25.536491 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e3f87b-4fb8-428e-a7d0-d343d38ae16e admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:25.551999 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1129/2276] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 9 12:15:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:15:25.556228 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44550522-a827-49ec-b6d5-9ede5c1468c1 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:25.572135 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44550522-a827-49ec-b6d5-9ede5c1468c1 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:25.577852 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9985a6b7-3f9b-4231-af21-b898b0baeb58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:25.578693 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9985a6b7-3f9b-4231-af21-b898b0baeb58 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:25.578830 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9985a6b7-3f9b-4231-af21-b898b0baeb58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:25.580754 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9985a6b7-3f9b-4231-af21-b898b0baeb58 admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:25.583245 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9985a6b7-3f9b-4231-af21-b898b0baeb58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:25.586776 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9985a6b7-3f9b-4231-af21-b898b0baeb58 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:25.640822 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1148/2277] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 9 12:15:25 2022] POST /identity/v3/projects => generated 373 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:15:25.654117 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e3f87b-4fb8-428e-a7d0-d343d38ae16e admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:25.672756 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e3f87b-4fb8-428e-a7d0-d343d38ae16e admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:25.744834 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf874d12-dbc6-46ff-a3f3-c5672f5212d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:25.745823 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf874d12-dbc6-46ff-a3f3-c5672f5212d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:25.746959 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf874d12-dbc6-46ff-a3f3-c5672f5212d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:25.747445 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf874d12-dbc6-46ff-a3f3-c5672f5212d4 admin admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:25.748402 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf874d12-dbc6-46ff-a3f3-c5672f5212d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:25.750541 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf874d12-dbc6-46ff-a3f3-c5672f5212d4 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:25.781799 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1130/2278] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:15:25 2022] POST /identity/v3/users => generated 287 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:15:25.786474 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9985a6b7-3f9b-4231-af21-b898b0baeb58 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:25.800907 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9985a6b7-3f9b-4231-af21-b898b0baeb58 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:25.809324 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6498eb34-bd6e-4191-85f6-1c0d4efb39d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:25.810184 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6498eb34-bd6e-4191-85f6-1c0d4efb39d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:25.810682 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6498eb34-bd6e-4191-85f6-1c0d4efb39d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:25.811011 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6498eb34-bd6e-4191-85f6-1c0d4efb39d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:25.811824 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6498eb34-bd6e-4191-85f6-1c0d4efb39d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:25.814413 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6498eb34-bd6e-4191-85f6-1c0d4efb39d8 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:25.834728 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf874d12-dbc6-46ff-a3f3-c5672f5212d4 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:25.835252 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1149/2279] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:15:25 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:15:25.845851 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf874d12-dbc6-46ff-a3f3-c5672f5212d4 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:25.852047 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c94022-1ca0-4dc0-95c3-3fe0145f65bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:25.852833 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c94022-1ca0-4dc0-95c3-3fe0145f65bb admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:25.853012 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c94022-1ca0-4dc0-95c3-3fe0145f65bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:25.853241 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c94022-1ca0-4dc0-95c3-3fe0145f65bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:25.853893 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c94022-1ca0-4dc0-95c3-3fe0145f65bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:25.855939 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c94022-1ca0-4dc0-95c3-3fe0145f65bb admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:25.867647 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1131/2280] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:15:25 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:15:25.871824 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6498eb34-bd6e-4191-85f6-1c0d4efb39d8 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:25.884034 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6498eb34-bd6e-4191-85f6-1c0d4efb39d8 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:25.895581 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea0b41f-f565-4767-b16d-c13890713977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:25.896273 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea0b41f-f565-4767-b16d-c13890713977 admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:25.896644 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea0b41f-f565-4767-b16d-c13890713977 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:25.897607 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea0b41f-f565-4767-b16d-c13890713977 admin admin] PATH_INFO: `/v3/projects/d544091be6564966a03dec95b456edcb/users/93bf854c92094200af66e51a690a3fd9/roles/eecd2f160dd844f5906f3d345780c501` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:25.914663 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea0b41f-f565-4767-b16d-c13890713977 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d544091be6564966a03dec95b456edcb, user_id=93bf854c92094200af66e51a690a3fd9, role_id=eecd2f160dd844f5906f3d345780c501)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:25.917539 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea0b41f-f565-4767-b16d-c13890713977 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:25.949017 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1150/2281] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:15:25 2022] PUT /identity/v3/projects/d544091be6564966a03dec95b456edcb/users/93bf854c92094200af66e51a690a3fd9/roles/eecd2f160dd844f5906f3d345780c501 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:15:25.959974 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a61b71-f21c-487a-b6fd-b60580ad0e88 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:25.960248 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a61b71-f21c-487a-b6fd-b60580ad0e88 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:25.960527 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a61b71-f21c-487a-b6fd-b60580ad0e88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:25.993399 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-f4a61b71-f21c-487a-b6fd-b60580ad0e88 None None] MFA Rules not processed for user `93bf854c92094200af66e51a690a3fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:15:25.994619 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-f4a61b71-f21c-487a-b6fd-b60580ad0e88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:15:26.067836 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1132/2282] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:15:25 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:15:26.084929 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea0b41f-f565-4767-b16d-c13890713977 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:26.086863 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-dea0b41f-f565-4767-b16d-c13890713977 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:15:26.151717 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:15:26.179749 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea0b41f-f565-4767-b16d-c13890713977 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:26.185333 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac1f190-c818-48f7-86c2-1844a2572444 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535c258100df40b58a5ee44b2c21d621', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:26.186243 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac1f190-c818-48f7-86c2-1844a2572444 service cinder] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:26.186691 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac1f190-c818-48f7-86c2-1844a2572444 service cinder] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:26.187274 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac1f190-c818-48f7-86c2-1844a2572444 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:26.198356 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac1f190-c818-48f7-86c2-1844a2572444 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:26.199844 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac1f190-c818-48f7-86c2-1844a2572444 service cinder] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:26.212212 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1151/2283] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:15:26 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:15:35.034107 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83964bba-da23-4e55-92a3-2904c6c60d74 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:35.034646 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83964bba-da23-4e55-92a3-2904c6c60d74 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:35.034646 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83964bba-da23-4e55-92a3-2904c6c60d74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:35.063934 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-83964bba-da23-4e55-92a3-2904c6c60d74 None None] MFA Rules not processed for user `31411178e5c342b7bb14a21dec62f540`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:15:35.065212 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-83964bba-da23-4e55-92a3-2904c6c60d74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:15:35.141498 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:15:35.185922 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1133/2284] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:15:35 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:15:36.206095 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=9 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:15:42.387784 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035bce57-f3b1-4e6b-a90a-3f857f331386 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:42.388386 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035bce57-f3b1-4e6b-a90a-3f857f331386 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:42.388386 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035bce57-f3b1-4e6b-a90a-3f857f331386 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:42.446000 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-035bce57-f3b1-4e6b-a90a-3f857f331386 None None] MFA Rules not processed for user `e368b4e668c54e25bff80a8e8a8c3588`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:15:42.454788 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-035bce57-f3b1-4e6b-a90a-3f857f331386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:15:42.562074 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83964bba-da23-4e55-92a3-2904c6c60d74 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:42.637468 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1152/2285] 10.222.0.49 () {58 vars in 1009 bytes} [Fri Sep 9 12:15:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:15:42.684791 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035bce57-f3b1-4e6b-a90a-3f857f331386 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:42.686461 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-035bce57-f3b1-4e6b-a90a-3f857f331386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:15:42.749611 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83964bba-da23-4e55-92a3-2904c6c60d74 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:42.767740 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a8b65d-0b85-438c-9b67-1a3e14e2272f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:42.768535 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a8b65d-0b85-438c-9b67-1a3e14e2272f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:42.768883 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a8b65d-0b85-438c-9b67-1a3e14e2272f admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:42.771195 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a8b65d-0b85-438c-9b67-1a3e14e2272f admin admin] PATH_INFO: `/v3/users/93bf854c92094200af66e51a690a3fd9` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:42.774109 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a8b65d-0b85-438c-9b67-1a3e14e2272f admin admin] RBAC: Authorizing `identity:delete_user(user_id=93bf854c92094200af66e51a690a3fd9)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:42.782180 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a8b65d-0b85-438c-9b67-1a3e14e2272f admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:42.836541 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035bce57-f3b1-4e6b-a90a-3f857f331386 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:42.848290 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75aecf7-14e1-43d9-a367-6c5b2f94faec service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e652ba827849b5a3fc1213d04a1b76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:42.849580 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75aecf7-14e1-43d9-a367-6c5b2f94faec service swift] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:42.850024 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75aecf7-14e1-43d9-a367-6c5b2f94faec service swift] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:42.850452 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75aecf7-14e1-43d9-a367-6c5b2f94faec service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:42.860813 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75aecf7-14e1-43d9-a367-6c5b2f94faec service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:42.862757 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75aecf7-14e1-43d9-a367-6c5b2f94faec service swift] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:42.880592 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1153/2286] 10.222.0.49 () {60 vars in 1384 bytes} [Fri Sep 9 12:15:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 09 12:15:42.925791 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-64a8b65d-0b85-438c-9b67-1a3e14e2272f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93bf854c92094200af66e51a690a3fd9'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:42.989005 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-64a8b65d-0b85-438c-9b67-1a3e14e2272f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93bf854c92094200af66e51a690a3fd9'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:43.002355 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-64a8b65d-0b85-438c-9b67-1a3e14e2272f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93bf854c92094200af66e51a690a3fd9'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:43.016247 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-64a8b65d-0b85-438c-9b67-1a3e14e2272f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93bf854c92094200af66e51a690a3fd9'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:43.017374 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-64a8b65d-0b85-438c-9b67-1a3e14e2272f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93bf854c92094200af66e51a690a3fd9'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:43.054481 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a00c98-7068-4a9d-9913-6a1a57de41b6 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:43.054792 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a00c98-7068-4a9d-9913-6a1a57de41b6 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:43.055226 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a00c98-7068-4a9d-9913-6a1a57de41b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:43.056087 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1134/2287] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 9 12:15:42 2022] DELETE /identity/v3/users/93bf854c92094200af66e51a690a3fd9 => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:15:43.124780 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-55a00c98-7068-4a9d-9913-6a1a57de41b6 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:15:43.132402 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-55a00c98-7068-4a9d-9913-6a1a57de41b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:15:43.314292 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1154/2288] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:15:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:15:43.349420 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a8b65d-0b85-438c-9b67-1a3e14e2272f admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:43.350482 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-64a8b65d-0b85-438c-9b67-1a3e14e2272f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:15:43.513014 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a8b65d-0b85-438c-9b67-1a3e14e2272f admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:43.529315 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0624486e-465e-4507-bdb6-ff077d44ce60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:43.533709 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0624486e-465e-4507-bdb6-ff077d44ce60 service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:43.534067 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0624486e-465e-4507-bdb6-ff077d44ce60 service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:43.534399 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0624486e-465e-4507-bdb6-ff077d44ce60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:43.576590 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0624486e-465e-4507-bdb6-ff077d44ce60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:43.578665 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0624486e-465e-4507-bdb6-ff077d44ce60 service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:43.585058 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:15:43.619454 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1135/2289] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:15:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:15:43.644622 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a00c98-7068-4a9d-9913-6a1a57de41b6 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:43.646121 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-55a00c98-7068-4a9d-9913-6a1a57de41b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:15:43.669710 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a00c98-7068-4a9d-9913-6a1a57de41b6 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:43.682428 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f75b5a-44c9-41b6-aa5c-2c8e82ec654a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:43.683396 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f75b5a-44c9-41b6-aa5c-2c8e82ec654a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:43.683678 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f75b5a-44c9-41b6-aa5c-2c8e82ec654a admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:43.683950 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f75b5a-44c9-41b6-aa5c-2c8e82ec654a admin admin] PATH_INFO: `/v3/users/324c01ff5dc24f75b74a61e7da3a5c42` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:43.685598 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f75b5a-44c9-41b6-aa5c-2c8e82ec654a admin admin] RBAC: Authorizing `identity:delete_user(user_id=324c01ff5dc24f75b74a61e7da3a5c42)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:43.687590 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f75b5a-44c9-41b6-aa5c-2c8e82ec654a admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:43.789917 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:15:43.812998 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-b6f75b5a-44c9-41b6-aa5c-2c8e82ec654a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '324c01ff5dc24f75b74a61e7da3a5c42'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:43.832425 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-b6f75b5a-44c9-41b6-aa5c-2c8e82ec654a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '324c01ff5dc24f75b74a61e7da3a5c42'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:43.867219 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-b6f75b5a-44c9-41b6-aa5c-2c8e82ec654a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '324c01ff5dc24f75b74a61e7da3a5c42'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:43.884331 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-b6f75b5a-44c9-41b6-aa5c-2c8e82ec654a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '324c01ff5dc24f75b74a61e7da3a5c42'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:43.885526 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-b6f75b5a-44c9-41b6-aa5c-2c8e82ec654a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '324c01ff5dc24f75b74a61e7da3a5c42'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:43.925897 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1155/2290] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 9 12:15:43 2022] DELETE /identity/v3/users/324c01ff5dc24f75b74a61e7da3a5c42 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:15:43.928561 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fe8891-b331-46d4-803b-f3ff424560fe None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:43.928839 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fe8891-b331-46d4-803b-f3ff424560fe None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:43.929121 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fe8891-b331-46d4-803b-f3ff424560fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:43.979991 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-a4fe8891-b331-46d4-803b-f3ff424560fe None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:15:43.983724 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-a4fe8891-b331-46d4-803b-f3ff424560fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:15:44.287461 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1136/2291] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:15:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 359 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:15:44.324949 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f75b5a-44c9-41b6-aa5c-2c8e82ec654a admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:44.326484 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-b6f75b5a-44c9-41b6-aa5c-2c8e82ec654a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:15:44.459287 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f75b5a-44c9-41b6-aa5c-2c8e82ec654a admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:44.468835 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb43f201-5d3c-45db-8632-08d9ae1dcdc9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:44.469530 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb43f201-5d3c-45db-8632-08d9ae1dcdc9 service neutron] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:44.469809 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb43f201-5d3c-45db-8632-08d9ae1dcdc9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:44.470082 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb43f201-5d3c-45db-8632-08d9ae1dcdc9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:44.496442 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb43f201-5d3c-45db-8632-08d9ae1dcdc9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:44.498717 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb43f201-5d3c-45db-8632-08d9ae1dcdc9 service neutron] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:44.525566 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1156/2292] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:15:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:15:45.760213 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fe8891-b331-46d4-803b-f3ff424560fe None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:45.761851 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-a4fe8891-b331-46d4-803b-f3ff424560fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:15:45.778931 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fe8891-b331-46d4-803b-f3ff424560fe None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:45.802499 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5791fce1-837c-4b2f-9898-07da3ae91fcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:45.803659 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5791fce1-837c-4b2f-9898-07da3ae91fcf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:45.804217 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5791fce1-837c-4b2f-9898-07da3ae91fcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:45.804787 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5791fce1-837c-4b2f-9898-07da3ae91fcf admin admin] PATH_INFO: `/v3/projects/46377e6c51234e3991205d996ea0ef5a` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:45.808662 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5791fce1-837c-4b2f-9898-07da3ae91fcf admin admin] RBAC: Authorizing `identity:delete_project(project_id=46377e6c51234e3991205d996ea0ef5a)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:45.813350 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5791fce1-837c-4b2f-9898-07da3ae91fcf admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:45.818222 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb43f201-5d3c-45db-8632-08d9ae1dcdc9 service neutron] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:45.820640 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-eb43f201-5d3c-45db-8632-08d9ae1dcdc9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:15:45.851199 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb43f201-5d3c-45db-8632-08d9ae1dcdc9 service neutron] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:45.873520 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c522f900-cda0-43f7-af90-a23d98e5fd75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:45.873785 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c522f900-cda0-43f7-af90-a23d98e5fd75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:45.874063 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c522f900-cda0-43f7-af90-a23d98e5fd75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:45.874334 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c522f900-cda0-43f7-af90-a23d98e5fd75 admin admin] PATH_INFO: `/v3/projects/d544091be6564966a03dec95b456edcb` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:45.878132 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c522f900-cda0-43f7-af90-a23d98e5fd75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d544091be6564966a03dec95b456edcb)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:45.882005 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c522f900-cda0-43f7-af90-a23d98e5fd75 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:45.955777 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-5791fce1-837c-4b2f-9898-07da3ae91fcf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46377e6c51234e3991205d996ea0ef5a', 'request_id': 'req-5791fce1-837c-4b2f-9898-07da3ae91fcf'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:45.964506 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-5791fce1-837c-4b2f-9898-07da3ae91fcf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46377e6c51234e3991205d996ea0ef5a', 'request_id': 'req-5791fce1-837c-4b2f-9898-07da3ae91fcf'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:45.976381 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-5791fce1-837c-4b2f-9898-07da3ae91fcf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46377e6c51234e3991205d996ea0ef5a', 'request_id': 'req-5791fce1-837c-4b2f-9898-07da3ae91fcf'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:45.996566 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-c522f900-cda0-43f7-af90-a23d98e5fd75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd544091be6564966a03dec95b456edcb', 'request_id': 'req-c522f900-cda0-43f7-af90-a23d98e5fd75'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:45.998755 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-5791fce1-837c-4b2f-9898-07da3ae91fcf admin admin] The token cache is being invalidate because project 46377e6c51234e3991205d996ea0ef5a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:15:45.999199 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-5791fce1-837c-4b2f-9898-07da3ae91fcf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:45.999971 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-5791fce1-837c-4b2f-9898-07da3ae91fcf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:46.006703 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1137/2293] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 9 12:15:45 2022] DELETE /identity/v3/projects/46377e6c51234e3991205d996ea0ef5a => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:15:46.008390 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec89b71-ce58-4ca6-86ed-0599e5dc347d None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:46.008886 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec89b71-ce58-4ca6-86ed-0599e5dc347d None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:46.009240 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec89b71-ce58-4ca6-86ed-0599e5dc347d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:46.021912 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-c522f900-cda0-43f7-af90-a23d98e5fd75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd544091be6564966a03dec95b456edcb', 'request_id': 'req-c522f900-cda0-43f7-af90-a23d98e5fd75'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:46.030445 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-c522f900-cda0-43f7-af90-a23d98e5fd75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd544091be6564966a03dec95b456edcb', 'request_id': 'req-c522f900-cda0-43f7-af90-a23d98e5fd75'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:46.038065 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-6ec89b71-ce58-4ca6-86ed-0599e5dc347d None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:15:46.041159 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-6ec89b71-ce58-4ca6-86ed-0599e5dc347d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:15:46.058562 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-c522f900-cda0-43f7-af90-a23d98e5fd75 admin admin] The token cache is being invalidate because project d544091be6564966a03dec95b456edcb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81591) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:15:46.058862 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-c522f900-cda0-43f7-af90-a23d98e5fd75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:46.059574 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-c522f900-cda0-43f7-af90-a23d98e5fd75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:46.074943 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1157/2294] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 9 12:15:45 2022] DELETE /identity/v3/projects/d544091be6564966a03dec95b456edcb => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:15:46.235015 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1138/2295] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:15:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:15:46.237481 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee936d2c-a8b2-482c-b71f-cf755241714f None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:46.237760 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee936d2c-a8b2-482c-b71f-cf755241714f None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:46.238030 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee936d2c-a8b2-482c-b71f-cf755241714f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:46.289060 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-ee936d2c-a8b2-482c-b71f-cf755241714f None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:15:46.295104 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-ee936d2c-a8b2-482c-b71f-cf755241714f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:15:46.336928 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec89b71-ce58-4ca6-86ed-0599e5dc347d None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:46.344034 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1158/2296] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:15:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:15:46.354676 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec89b71-ce58-4ca6-86ed-0599e5dc347d None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:46.360503 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793affc6-0c72-4a3d-970d-848fa9376957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:46.361389 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793affc6-0c72-4a3d-970d-848fa9376957 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:46.361912 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793affc6-0c72-4a3d-970d-848fa9376957 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:46.362398 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793affc6-0c72-4a3d-970d-848fa9376957 admin admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:46.363287 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793affc6-0c72-4a3d-970d-848fa9376957 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:15:46.363792 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793affc6-0c72-4a3d-970d-848fa9376957 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:46.366480 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793affc6-0c72-4a3d-970d-848fa9376957 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:46.390261 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1139/2297] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 9 12:15:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:15:46.391199 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee936d2c-a8b2-482c-b71f-cf755241714f None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:46.402336 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee936d2c-a8b2-482c-b71f-cf755241714f None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:46.408353 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3fafe2-0517-4c40-8099-711992920cd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:46.409537 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3fafe2-0517-4c40-8099-711992920cd7 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:46.410049 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3fafe2-0517-4c40-8099-711992920cd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:46.410533 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3fafe2-0517-4c40-8099-711992920cd7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:46.411457 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3fafe2-0517-4c40-8099-711992920cd7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:46.414123 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3fafe2-0517-4c40-8099-711992920cd7 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:46.452343 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1159/2298] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 9 12:15:46 2022] POST /identity/v3/projects => generated 365 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:15:46.452964 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793affc6-0c72-4a3d-970d-848fa9376957 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:46.467394 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793affc6-0c72-4a3d-970d-848fa9376957 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:46.474225 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:15:46.579618 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b563eff-c337-45f2-b655-2688d6c85d29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:46.580234 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b563eff-c337-45f2-b655-2688d6c85d29 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:46.580508 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b563eff-c337-45f2-b655-2688d6c85d29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:46.580779 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b563eff-c337-45f2-b655-2688d6c85d29 admin admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:46.581410 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b563eff-c337-45f2-b655-2688d6c85d29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:46.584500 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b563eff-c337-45f2-b655-2688d6c85d29 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:46.618620 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3fafe2-0517-4c40-8099-711992920cd7 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:46.621110 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1140/2299] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:15:46 2022] POST /identity/v3/users => generated 283 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:15:46.633807 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:15:46.639955 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3fafe2-0517-4c40-8099-711992920cd7 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:46.657393 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685bbc51-bdd3-4fe8-a508-b5a94ea7c32f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:46.658441 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685bbc51-bdd3-4fe8-a508-b5a94ea7c32f admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:46.658441 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685bbc51-bdd3-4fe8-a508-b5a94ea7c32f admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:46.658628 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685bbc51-bdd3-4fe8-a508-b5a94ea7c32f admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:46.659445 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685bbc51-bdd3-4fe8-a508-b5a94ea7c32f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:46.661989 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685bbc51-bdd3-4fe8-a508-b5a94ea7c32f admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:46.675151 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1160/2300] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:15:46 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:15:46.680034 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b563eff-c337-45f2-b655-2688d6c85d29 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:46.697372 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b563eff-c337-45f2-b655-2688d6c85d29 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:46.703402 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27da5d8d-f9ee-4d2a-8206-18edb9fbd466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:46.704904 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27da5d8d-f9ee-4d2a-8206-18edb9fbd466 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:46.705303 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27da5d8d-f9ee-4d2a-8206-18edb9fbd466 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:46.705592 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27da5d8d-f9ee-4d2a-8206-18edb9fbd466 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:46.706251 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27da5d8d-f9ee-4d2a-8206-18edb9fbd466 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:46.709014 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27da5d8d-f9ee-4d2a-8206-18edb9fbd466 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:46.746792 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685bbc51-bdd3-4fe8-a508-b5a94ea7c32f admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:46.757919 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1141/2301] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:15:46 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:15:46.763809 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685bbc51-bdd3-4fe8-a508-b5a94ea7c32f admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:46.775782 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee2c03e-e190-4b33-91bf-f66cc6d0fef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:46.776490 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee2c03e-e190-4b33-91bf-f66cc6d0fef6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:46.776877 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee2c03e-e190-4b33-91bf-f66cc6d0fef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:46.777223 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee2c03e-e190-4b33-91bf-f66cc6d0fef6 admin admin] PATH_INFO: `/v3/projects/b37dde899da24a69b858c42c4f499b16/users/bf11352fb94e4762865702ddd3dc73fe/roles/eecd2f160dd844f5906f3d345780c501` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:46.811893 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee2c03e-e190-4b33-91bf-f66cc6d0fef6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b37dde899da24a69b858c42c4f499b16, user_id=bf11352fb94e4762865702ddd3dc73fe, role_id=eecd2f160dd844f5906f3d345780c501)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:46.814782 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee2c03e-e190-4b33-91bf-f66cc6d0fef6 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:46.855159 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1161/2302] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:15:46 2022] PUT /identity/v3/projects/b37dde899da24a69b858c42c4f499b16/users/bf11352fb94e4762865702ddd3dc73fe/roles/eecd2f160dd844f5906f3d345780c501 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:15:46.869851 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59365e5e-28f3-4746-8950-f9dab4912273 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:46.870436 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59365e5e-28f3-4746-8950-f9dab4912273 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:46.871233 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59365e5e-28f3-4746-8950-f9dab4912273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:46.950865 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-59365e5e-28f3-4746-8950-f9dab4912273 None None] MFA Rules not processed for user `bf11352fb94e4762865702ddd3dc73fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:15:46.952125 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-59365e5e-28f3-4746-8950-f9dab4912273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:15:47.101180 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1142/2303] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:15:46 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:15:47.113800 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee2c03e-e190-4b33-91bf-f66cc6d0fef6 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:47.115954 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-7ee2c03e-e190-4b33-91bf-f66cc6d0fef6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:15:47.222136 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee2c03e-e190-4b33-91bf-f66cc6d0fef6 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:47.232096 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebcaf1f-ab92-496a-ae02-c861245648cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535c258100df40b58a5ee44b2c21d621', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:47.232945 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebcaf1f-ab92-496a-ae02-c861245648cd service cinder] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:47.233392 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebcaf1f-ab92-496a-ae02-c861245648cd service cinder] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:47.233850 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebcaf1f-ab92-496a-ae02-c861245648cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:47.250390 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebcaf1f-ab92-496a-ae02-c861245648cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:47.252961 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebcaf1f-ab92-496a-ae02-c861245648cd service cinder] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:47.273917 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1162/2304] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:15:47 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:15:47.285186 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfe08b3-8b6d-4fe7-a326-8751ff96b6ae None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:47.285675 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfe08b3-8b6d-4fe7-a326-8751ff96b6ae None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:47.286835 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfe08b3-8b6d-4fe7-a326-8751ff96b6ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:47.324513 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-ddfe08b3-8b6d-4fe7-a326-8751ff96b6ae None None] MFA Rules not processed for user `e368b4e668c54e25bff80a8e8a8c3588`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:15:47.325603 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-ddfe08b3-8b6d-4fe7-a326-8751ff96b6ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:15:47.513145 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1143/2305] 10.222.0.49 () {58 vars in 967 bytes} [Fri Sep 9 12:15:47 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:15:47.527255 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebcaf1f-ab92-496a-ae02-c861245648cd service cinder] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:47.528701 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-aebcaf1f-ab92-496a-ae02-c861245648cd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:15:47.728503 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebcaf1f-ab92-496a-ae02-c861245648cd service cinder] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:47.742525 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53791ee6-2986-4660-98a0-807faaf08637 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e652ba827849b5a3fc1213d04a1b76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:47.743430 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53791ee6-2986-4660-98a0-807faaf08637 service swift] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:47.743673 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53791ee6-2986-4660-98a0-807faaf08637 service swift] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:47.743949 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53791ee6-2986-4660-98a0-807faaf08637 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:47.768596 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53791ee6-2986-4660-98a0-807faaf08637 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:47.772887 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53791ee6-2986-4660-98a0-807faaf08637 service swift] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:47.799607 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1163/2306] 10.222.0.49 () {60 vars in 1384 bytes} [Fri Sep 9 12:15:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 12:15:50.338643 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfe08b3-8b6d-4fe7-a326-8751ff96b6ae None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:50.342089 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-ddfe08b3-8b6d-4fe7-a326-8751ff96b6ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:15:50.407020 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:15:50.489838 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfe08b3-8b6d-4fe7-a326-8751ff96b6ae None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:50.508046 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0689eeb3-86b6-454f-a464-79b92eae89b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:50.509121 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0689eeb3-86b6-454f-a464-79b92eae89b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:50.509602 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0689eeb3-86b6-454f-a464-79b92eae89b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:50.511218 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0689eeb3-86b6-454f-a464-79b92eae89b7 admin admin] PATH_INFO: `/v3/users/913344053b6244c584140807ab2149cb` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:50.512773 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0689eeb3-86b6-454f-a464-79b92eae89b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=913344053b6244c584140807ab2149cb)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:50.518387 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0689eeb3-86b6-454f-a464-79b92eae89b7 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:50.612618 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-0689eeb3-86b6-454f-a464-79b92eae89b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '913344053b6244c584140807ab2149cb'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:50.631147 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-0689eeb3-86b6-454f-a464-79b92eae89b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '913344053b6244c584140807ab2149cb'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:50.645755 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-0689eeb3-86b6-454f-a464-79b92eae89b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '913344053b6244c584140807ab2149cb'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:50.660003 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-0689eeb3-86b6-454f-a464-79b92eae89b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '913344053b6244c584140807ab2149cb'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:50.662610 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-0689eeb3-86b6-454f-a464-79b92eae89b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '913344053b6244c584140807ab2149cb'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:50.680163 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1144/2307] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 9 12:15:50 2022] DELETE /identity/v3/users/913344053b6244c584140807ab2149cb => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:15:50.955858 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53791ee6-2986-4660-98a0-807faaf08637 service swift] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:50.979165 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53791ee6-2986-4660-98a0-807faaf08637 service swift] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:51.053952 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268cd242-5bf0-4d70-8d02-b78ccc30d415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:51.054931 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268cd242-5bf0-4d70-8d02-b78ccc30d415 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:51.055226 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268cd242-5bf0-4d70-8d02-b78ccc30d415 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:51.055504 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268cd242-5bf0-4d70-8d02-b78ccc30d415 admin admin] PATH_INFO: `/v3/projects/80d60a0051c24f9b98bc252a0d0754cc` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:51.057266 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268cd242-5bf0-4d70-8d02-b78ccc30d415 admin admin] RBAC: Authorizing `identity:delete_project(project_id=80d60a0051c24f9b98bc252a0d0754cc)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:51.059744 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268cd242-5bf0-4d70-8d02-b78ccc30d415 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:51.063354 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:15:51.126960 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-268cd242-5bf0-4d70-8d02-b78ccc30d415 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80d60a0051c24f9b98bc252a0d0754cc', 'request_id': 'req-268cd242-5bf0-4d70-8d02-b78ccc30d415'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:51.137993 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-268cd242-5bf0-4d70-8d02-b78ccc30d415 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80d60a0051c24f9b98bc252a0d0754cc', 'request_id': 'req-268cd242-5bf0-4d70-8d02-b78ccc30d415'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:51.154571 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-268cd242-5bf0-4d70-8d02-b78ccc30d415 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80d60a0051c24f9b98bc252a0d0754cc', 'request_id': 'req-268cd242-5bf0-4d70-8d02-b78ccc30d415'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:51.173285 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-268cd242-5bf0-4d70-8d02-b78ccc30d415 admin admin] The token cache is being invalidate because project 80d60a0051c24f9b98bc252a0d0754cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81591) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:15:51.174649 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-268cd242-5bf0-4d70-8d02-b78ccc30d415 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:51.175930 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-268cd242-5bf0-4d70-8d02-b78ccc30d415 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:51.180473 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1164/2308] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 9 12:15:50 2022] DELETE /identity/v3/projects/80d60a0051c24f9b98bc252a0d0754cc => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:15:51.184804 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fccc71a-be65-4102-874b-47dc6b421e1c None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:51.185156 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fccc71a-be65-4102-874b-47dc6b421e1c None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:51.185462 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fccc71a-be65-4102-874b-47dc6b421e1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:51.210504 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-6fccc71a-be65-4102-874b-47dc6b421e1c None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:15:51.211689 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-6fccc71a-be65-4102-874b-47dc6b421e1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:15:51.342377 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1145/2309] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:15:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:15:51.350857 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0192e4a-5fcc-42b9-825e-631392fe06d5 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:51.351522 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0192e4a-5fcc-42b9-825e-631392fe06d5 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:51.352007 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0192e4a-5fcc-42b9-825e-631392fe06d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:51.388808 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-b0192e4a-5fcc-42b9-825e-631392fe06d5 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:15:51.390307 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-b0192e4a-5fcc-42b9-825e-631392fe06d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:15:51.405649 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1165/2310] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:15:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:15:51.409708 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fccc71a-be65-4102-874b-47dc6b421e1c None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:51.423956 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fccc71a-be65-4102-874b-47dc6b421e1c None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:51.430701 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b60d778-d89e-4849-a877-3d1193814922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:51.431365 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b60d778-d89e-4849-a877-3d1193814922 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:51.431645 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b60d778-d89e-4849-a877-3d1193814922 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:51.432057 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b60d778-d89e-4849-a877-3d1193814922 admin admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:51.433047 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b60d778-d89e-4849-a877-3d1193814922 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:15:51.433356 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b60d778-d89e-4849-a877-3d1193814922 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:51.435079 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b60d778-d89e-4849-a877-3d1193814922 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:51.452904 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1146/2311] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 9 12:15:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:15:51.453712 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0192e4a-5fcc-42b9-825e-631392fe06d5 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:51.464563 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0192e4a-5fcc-42b9-825e-631392fe06d5 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:51.469664 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036ebaa0-92ed-423f-8e55-76b4190bb881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:51.470347 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036ebaa0-92ed-423f-8e55-76b4190bb881 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:51.470695 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036ebaa0-92ed-423f-8e55-76b4190bb881 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:51.471187 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036ebaa0-92ed-423f-8e55-76b4190bb881 admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:51.471885 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036ebaa0-92ed-423f-8e55-76b4190bb881 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:51.474663 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036ebaa0-92ed-423f-8e55-76b4190bb881 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:51.518956 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1166/2312] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 9 12:15:51 2022] POST /identity/v3/projects => generated 371 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:15:51.521788 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b60d778-d89e-4849-a877-3d1193814922 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:51.537745 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b60d778-d89e-4849-a877-3d1193814922 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:51.604376 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:15:51.654035 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d48b5e-be23-437b-b172-fed218a000d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:51.654783 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d48b5e-be23-437b-b172-fed218a000d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:51.655185 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d48b5e-be23-437b-b172-fed218a000d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:51.655555 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d48b5e-be23-437b-b172-fed218a000d0 admin admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:51.656294 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d48b5e-be23-437b-b172-fed218a000d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:51.658327 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d48b5e-be23-437b-b172-fed218a000d0 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:51.701950 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1147/2313] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:15:51 2022] POST /identity/v3/users => generated 286 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:15:51.703312 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036ebaa0-92ed-423f-8e55-76b4190bb881 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:51.722490 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036ebaa0-92ed-423f-8e55-76b4190bb881 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:51.730397 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9bc753-e5bf-4ba5-812e-c90454755840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:51.731446 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9bc753-e5bf-4ba5-812e-c90454755840 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:51.731827 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9bc753-e5bf-4ba5-812e-c90454755840 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:51.732682 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9bc753-e5bf-4ba5-812e-c90454755840 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:51.732974 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9bc753-e5bf-4ba5-812e-c90454755840 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:51.734920 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9bc753-e5bf-4ba5-812e-c90454755840 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:51.751832 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1167/2314] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:15:51 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:15:51.754953 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d48b5e-be23-437b-b172-fed218a000d0 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:51.770476 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d48b5e-be23-437b-b172-fed218a000d0 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:51.782792 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564a1ecd-604d-4e94-9312-4e22bc8c607b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:51.783496 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564a1ecd-604d-4e94-9312-4e22bc8c607b admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:51.783778 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564a1ecd-604d-4e94-9312-4e22bc8c607b admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:51.784053 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564a1ecd-604d-4e94-9312-4e22bc8c607b admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:51.784710 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564a1ecd-604d-4e94-9312-4e22bc8c607b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:51.788331 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564a1ecd-604d-4e94-9312-4e22bc8c607b admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:51.815018 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1148/2315] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:15:51 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:15:51.820664 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9bc753-e5bf-4ba5-812e-c90454755840 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:51.834263 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9bc753-e5bf-4ba5-812e-c90454755840 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:51.849912 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9654db7-ba43-4895-802a-e09ecee29053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:51.851940 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9654db7-ba43-4895-802a-e09ecee29053 admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:51.852498 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9654db7-ba43-4895-802a-e09ecee29053 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:51.853182 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9654db7-ba43-4895-802a-e09ecee29053 admin admin] PATH_INFO: `/v3/projects/871f861c89644686849527ca9484f02b/users/6b31c44b37174833a3945f26a6cac183/roles/eecd2f160dd844f5906f3d345780c501` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:51.885820 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9654db7-ba43-4895-802a-e09ecee29053 admin admin] RBAC: Authorizing `identity:create_grant(project_id=871f861c89644686849527ca9484f02b, user_id=6b31c44b37174833a3945f26a6cac183, role_id=eecd2f160dd844f5906f3d345780c501)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:51.888744 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9654db7-ba43-4895-802a-e09ecee29053 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:51.909115 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1168/2316] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:15:51 2022] PUT /identity/v3/projects/871f861c89644686849527ca9484f02b/users/6b31c44b37174833a3945f26a6cac183/roles/eecd2f160dd844f5906f3d345780c501 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:15:51.920100 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f24931-d223-4c4b-9a73-28ee9530ba5b None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:51.920849 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f24931-d223-4c4b-9a73-28ee9530ba5b None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:51.921396 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f24931-d223-4c4b-9a73-28ee9530ba5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:51.954880 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-e7f24931-d223-4c4b-9a73-28ee9530ba5b None None] MFA Rules not processed for user `6b31c44b37174833a3945f26a6cac183`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:15:51.956528 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-e7f24931-d223-4c4b-9a73-28ee9530ba5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:15:52.055496 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1149/2317] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:15:51 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:15:52.066463 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9654db7-ba43-4895-802a-e09ecee29053 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:52.069664 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-e9654db7-ba43-4895-802a-e09ecee29053 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:15:52.170980 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9654db7-ba43-4895-802a-e09ecee29053 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:52.177022 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff85416-ce8a-4af2-bcfb-e5c8509a231c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535c258100df40b58a5ee44b2c21d621', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:52.177903 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff85416-ce8a-4af2-bcfb-e5c8509a231c service cinder] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:52.178336 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff85416-ce8a-4af2-bcfb-e5c8509a231c service cinder] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:52.178789 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff85416-ce8a-4af2-bcfb-e5c8509a231c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:52.190142 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff85416-ce8a-4af2-bcfb-e5c8509a231c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:52.191760 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff85416-ce8a-4af2-bcfb-e5c8509a231c service cinder] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:52.201883 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1169/2318] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:15:52 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:15:52.229778 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f24931-d223-4c4b-9a73-28ee9530ba5b None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:52.305503 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f24931-d223-4c4b-9a73-28ee9530ba5b None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:52.310809 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b65573-eeb4-4efe-bd2e-4335045f6ee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:52.311525 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b65573-eeb4-4efe-bd2e-4335045f6ee6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:52.312537 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b65573-eeb4-4efe-bd2e-4335045f6ee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:52.312826 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b65573-eeb4-4efe-bd2e-4335045f6ee6 admin admin] PATH_INFO: `/v3/users/6b31c44b37174833a3945f26a6cac183` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:52.314207 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b65573-eeb4-4efe-bd2e-4335045f6ee6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b31c44b37174833a3945f26a6cac183)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:52.316267 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b65573-eeb4-4efe-bd2e-4335045f6ee6 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:52.391001 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:15:52.400079 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-43b65573-eeb4-4efe-bd2e-4335045f6ee6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b31c44b37174833a3945f26a6cac183'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:52.417943 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-43b65573-eeb4-4efe-bd2e-4335045f6ee6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b31c44b37174833a3945f26a6cac183'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:52.431169 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-43b65573-eeb4-4efe-bd2e-4335045f6ee6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b31c44b37174833a3945f26a6cac183'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:52.443261 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-43b65573-eeb4-4efe-bd2e-4335045f6ee6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b31c44b37174833a3945f26a6cac183'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:52.444168 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-43b65573-eeb4-4efe-bd2e-4335045f6ee6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b31c44b37174833a3945f26a6cac183'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:52.452401 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1150/2319] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 9 12:15:52 2022] DELETE /identity/v3/users/6b31c44b37174833a3945f26a6cac183 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:15:52.459215 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7cac0a-8394-4f0a-bcc9-b841af5cb5f3 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:52.459891 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7cac0a-8394-4f0a-bcc9-b841af5cb5f3 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:52.460869 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7cac0a-8394-4f0a-bcc9-b841af5cb5f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:52.494584 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-3c7cac0a-8394-4f0a-bcc9-b841af5cb5f3 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:15:52.495884 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-3c7cac0a-8394-4f0a-bcc9-b841af5cb5f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:15:52.532034 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:15:52.590849 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1170/2320] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:15:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:15:52.604552 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b65573-eeb4-4efe-bd2e-4335045f6ee6 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:52.610301 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-43b65573-eeb4-4efe-bd2e-4335045f6ee6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:15:52.774996 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b65573-eeb4-4efe-bd2e-4335045f6ee6 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:52.789060 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c1f03a-8bb4-4c32-93c8-f318439d0890 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:52.790120 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c1f03a-8bb4-4c32-93c8-f318439d0890 service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:52.790309 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c1f03a-8bb4-4c32-93c8-f318439d0890 service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:52.790764 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c1f03a-8bb4-4c32-93c8-f318439d0890 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:52.812751 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c1f03a-8bb4-4c32-93c8-f318439d0890 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:52.814553 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c1f03a-8bb4-4c32-93c8-f318439d0890 service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:52.829805 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1151/2321] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:15:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:15:53.533735 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7cac0a-8394-4f0a-bcc9-b841af5cb5f3 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:53.535881 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-3c7cac0a-8394-4f0a-bcc9-b841af5cb5f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:15:53.575007 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7cac0a-8394-4f0a-bcc9-b841af5cb5f3 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:53.603944 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba5b973-4418-42e9-b7d0-bd052272aea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:53.604794 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba5b973-4418-42e9-b7d0-bd052272aea5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:53.605242 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba5b973-4418-42e9-b7d0-bd052272aea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:53.605661 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba5b973-4418-42e9-b7d0-bd052272aea5 admin admin] PATH_INFO: `/v3/projects/871f861c89644686849527ca9484f02b` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:53.608820 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba5b973-4418-42e9-b7d0-bd052272aea5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=871f861c89644686849527ca9484f02b)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:53.610921 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba5b973-4418-42e9-b7d0-bd052272aea5 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:53.709658 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-bba5b973-4418-42e9-b7d0-bd052272aea5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '871f861c89644686849527ca9484f02b', 'request_id': 'req-bba5b973-4418-42e9-b7d0-bd052272aea5'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:53.717529 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-bba5b973-4418-42e9-b7d0-bd052272aea5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '871f861c89644686849527ca9484f02b', 'request_id': 'req-bba5b973-4418-42e9-b7d0-bd052272aea5'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:53.727045 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-bba5b973-4418-42e9-b7d0-bd052272aea5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '871f861c89644686849527ca9484f02b', 'request_id': 'req-bba5b973-4418-42e9-b7d0-bd052272aea5'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:53.750773 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-bba5b973-4418-42e9-b7d0-bd052272aea5 admin admin] The token cache is being invalidate because project 871f861c89644686849527ca9484f02b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81591) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:15:53.751320 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-bba5b973-4418-42e9-b7d0-bd052272aea5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:53.752131 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-bba5b973-4418-42e9-b7d0-bd052272aea5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:15:53.754184 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1171/2322] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 9 12:15:53 2022] DELETE /identity/v3/projects/871f861c89644686849527ca9484f02b => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:15:53.759416 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da34d4b-c382-49f0-97c3-1d512e9aa003 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:53.760149 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da34d4b-c382-49f0-97c3-1d512e9aa003 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:53.760149 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da34d4b-c382-49f0-97c3-1d512e9aa003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:53.782770 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-4da34d4b-c382-49f0-97c3-1d512e9aa003 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:15:53.784020 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-4da34d4b-c382-49f0-97c3-1d512e9aa003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:15:53.858819 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1152/2323] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:15:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:15:53.868459 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda16eaa-dd0e-4c8e-b7c3-87271dadb3f8 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:53.868668 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda16eaa-dd0e-4c8e-b7c3-87271dadb3f8 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:53.868913 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda16eaa-dd0e-4c8e-b7c3-87271dadb3f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:53.892088 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-cda16eaa-dd0e-4c8e-b7c3-87271dadb3f8 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:15:53.893929 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-cda16eaa-dd0e-4c8e-b7c3-87271dadb3f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:15:53.902986 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1172/2324] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:15:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:15:53.907702 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da34d4b-c382-49f0-97c3-1d512e9aa003 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:53.919928 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da34d4b-c382-49f0-97c3-1d512e9aa003 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:53.927752 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299f59e2-9a97-467b-8a7f-1bbf831c0e34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:53.928563 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299f59e2-9a97-467b-8a7f-1bbf831c0e34 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:53.929278 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299f59e2-9a97-467b-8a7f-1bbf831c0e34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:53.929737 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299f59e2-9a97-467b-8a7f-1bbf831c0e34 admin admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:53.930560 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299f59e2-9a97-467b-8a7f-1bbf831c0e34 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:15:53.931655 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299f59e2-9a97-467b-8a7f-1bbf831c0e34 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:53.934104 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299f59e2-9a97-467b-8a7f-1bbf831c0e34 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:53.952579 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1153/2325] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 9 12:15:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:15:53.953668 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda16eaa-dd0e-4c8e-b7c3-87271dadb3f8 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:53.965033 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda16eaa-dd0e-4c8e-b7c3-87271dadb3f8 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:53.970654 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8502d778-05d0-4db6-bcc8-d995163298f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:53.971296 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8502d778-05d0-4db6-bcc8-d995163298f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:53.971570 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8502d778-05d0-4db6-bcc8-d995163298f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:53.971840 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8502d778-05d0-4db6-bcc8-d995163298f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:53.972478 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8502d778-05d0-4db6-bcc8-d995163298f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:53.975591 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8502d778-05d0-4db6-bcc8-d995163298f5 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:54.021188 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1173/2326] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 9 12:15:53 2022] POST /identity/v3/projects => generated 383 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:15:54.021531 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299f59e2-9a97-467b-8a7f-1bbf831c0e34 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:54.036089 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299f59e2-9a97-467b-8a7f-1bbf831c0e34 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:54.107224 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:15:54.126603 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09312b0f-1365-458e-82e7-407ed35ae7a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:54.127269 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09312b0f-1365-458e-82e7-407ed35ae7a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:54.127548 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09312b0f-1365-458e-82e7-407ed35ae7a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:54.127849 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09312b0f-1365-458e-82e7-407ed35ae7a9 admin admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:54.128478 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09312b0f-1365-458e-82e7-407ed35ae7a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:54.131241 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09312b0f-1365-458e-82e7-407ed35ae7a9 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:54.173503 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1154/2327] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:15:54 2022] POST /identity/v3/users => generated 292 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:15:54.179360 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8502d778-05d0-4db6-bcc8-d995163298f5 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:54.195869 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8502d778-05d0-4db6-bcc8-d995163298f5 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:54.208333 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9618d72f-411a-40f5-8f58-16539caa7435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:54.209324 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9618d72f-411a-40f5-8f58-16539caa7435 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:54.209720 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9618d72f-411a-40f5-8f58-16539caa7435 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:54.210057 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9618d72f-411a-40f5-8f58-16539caa7435 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:54.211486 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9618d72f-411a-40f5-8f58-16539caa7435 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:54.215372 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9618d72f-411a-40f5-8f58-16539caa7435 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:54.255211 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1174/2328] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:15:54 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 12:15:54.256498 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09312b0f-1365-458e-82e7-407ed35ae7a9 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:54.279653 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09312b0f-1365-458e-82e7-407ed35ae7a9 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:54.284803 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884b6d37-3082-4313-a1da-a1b3979213d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:54.287235 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884b6d37-3082-4313-a1da-a1b3979213d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:54.287574 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884b6d37-3082-4313-a1da-a1b3979213d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:54.287921 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884b6d37-3082-4313-a1da-a1b3979213d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:54.288647 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884b6d37-3082-4313-a1da-a1b3979213d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:54.291064 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884b6d37-3082-4313-a1da-a1b3979213d8 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:54.309815 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1155/2329] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:15:54 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:15:54.313980 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9618d72f-411a-40f5-8f58-16539caa7435 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:54.337092 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9618d72f-411a-40f5-8f58-16539caa7435 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:54.352402 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45859a91-ac00-4a67-9027-1078f9109165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:54.354430 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45859a91-ac00-4a67-9027-1078f9109165 admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:54.354946 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45859a91-ac00-4a67-9027-1078f9109165 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:54.355445 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45859a91-ac00-4a67-9027-1078f9109165 admin admin] PATH_INFO: `/v3/projects/baec172d59054da499387e85a4b8727e/users/eb42a8cde3044a4b9df5cb81d82d9910/roles/eecd2f160dd844f5906f3d345780c501` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:54.382756 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45859a91-ac00-4a67-9027-1078f9109165 admin admin] RBAC: Authorizing `identity:create_grant(project_id=baec172d59054da499387e85a4b8727e, user_id=eb42a8cde3044a4b9df5cb81d82d9910, role_id=eecd2f160dd844f5906f3d345780c501)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:54.387031 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45859a91-ac00-4a67-9027-1078f9109165 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:54.417597 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1175/2330] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:15:54 2022] PUT /identity/v3/projects/baec172d59054da499387e85a4b8727e/users/eb42a8cde3044a4b9df5cb81d82d9910/roles/eecd2f160dd844f5906f3d345780c501 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:15:54.420126 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8455ce62-af6b-4e92-bedb-d5b2f008b98e None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:54.420376 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8455ce62-af6b-4e92-bedb-d5b2f008b98e None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:54.420486 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8455ce62-af6b-4e92-bedb-d5b2f008b98e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:54.451884 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-8455ce62-af6b-4e92-bedb-d5b2f008b98e None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:15:54.453412 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-8455ce62-af6b-4e92-bedb-d5b2f008b98e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:15:54.496772 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45859a91-ac00-4a67-9027-1078f9109165 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:54.498640 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-45859a91-ac00-4a67-9027-1078f9109165 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:15:54.583772 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1156/2331] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:15:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:15:54.596106 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8455ce62-af6b-4e92-bedb-d5b2f008b98e None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:15:54.597705 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45859a91-ac00-4a67-9027-1078f9109165 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:54.598358 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-8455ce62-af6b-4e92-bedb-d5b2f008b98e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:15:54.604157 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a34add1-86b7-458a-90d8-7a9ba1b092af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:54.604855 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a34add1-86b7-458a-90d8-7a9ba1b092af service neutron] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:54.605091 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a34add1-86b7-458a-90d8-7a9ba1b092af service neutron] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:54.605590 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a34add1-86b7-458a-90d8-7a9ba1b092af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:54.607894 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-2a34add1-86b7-458a-90d8-7a9ba1b092af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:15:54.614412 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:15:54.623558 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8455ce62-af6b-4e92-bedb-d5b2f008b98e None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:15:54.636604 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210890a5-60fc-4976-aacc-a3fcee39f46f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:15:54.637148 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210890a5-60fc-4976-aacc-a3fcee39f46f service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:15:54.637485 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210890a5-60fc-4976-aacc-a3fcee39f46f service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:15:54.638043 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210890a5-60fc-4976-aacc-a3fcee39f46f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:15:54.661416 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210890a5-60fc-4976-aacc-a3fcee39f46f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:54.663077 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210890a5-60fc-4976-aacc-a3fcee39f46f service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:54.695492 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1157/2332] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:15:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:15:54.745249 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a34add1-86b7-458a-90d8-7a9ba1b092af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:15:54.748300 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a34add1-86b7-458a-90d8-7a9ba1b092af service neutron] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:15:54.774564 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1176/2333] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:15:54 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:16:01.408889 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b8191d-833d-4aa0-8d09-9d620ff98589 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:01.409459 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b8191d-833d-4aa0-8d09-9d620ff98589 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:01.409459 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b8191d-833d-4aa0-8d09-9d620ff98589 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:01.438871 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-b7b8191d-833d-4aa0-8d09-9d620ff98589 None None] MFA Rules not processed for user `eb42a8cde3044a4b9df5cb81d82d9910`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:16:01.440165 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-b7b8191d-833d-4aa0-8d09-9d620ff98589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:01.511912 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1158/2334] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:16:01 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:16:01.513404 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a34add1-86b7-458a-90d8-7a9ba1b092af service neutron] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:01.524710 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a34add1-86b7-458a-90d8-7a9ba1b092af service neutron] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:01.529837 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d590120-7047-420e-b961-4e9c12fc0912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:01.530448 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d590120-7047-420e-b961-4e9c12fc0912 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:01.530726 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d590120-7047-420e-b961-4e9c12fc0912 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:01.531029 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d590120-7047-420e-b961-4e9c12fc0912 admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:01.531675 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d590120-7047-420e-b961-4e9c12fc0912 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:01.533583 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d590120-7047-420e-b961-4e9c12fc0912 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:01.564943 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1177/2335] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 9 12:16:01 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:16:01.566888 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b8191d-833d-4aa0-8d09-9d620ff98589 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:01.578987 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b8191d-833d-4aa0-8d09-9d620ff98589 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:01.620019 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:16:01.639930 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73c85c3-911e-45c7-800f-c442c4520735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:01.640647 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73c85c3-911e-45c7-800f-c442c4520735 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:01.640933 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73c85c3-911e-45c7-800f-c442c4520735 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:01.641208 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73c85c3-911e-45c7-800f-c442c4520735 admin admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:01.641843 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73c85c3-911e-45c7-800f-c442c4520735 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:01.643900 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73c85c3-911e-45c7-800f-c442c4520735 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:01.666689 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1159/2336] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:16:01 2022] POST /identity/v3/users => generated 299 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:16:01.671234 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d590120-7047-420e-b961-4e9c12fc0912 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:01.684447 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d590120-7047-420e-b961-4e9c12fc0912 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:01.690422 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046d5c27-f2c0-4306-b873-2373a17f1fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:01.691171 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046d5c27-f2c0-4306-b873-2373a17f1fde admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:01.691549 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046d5c27-f2c0-4306-b873-2373a17f1fde admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:01.691938 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046d5c27-f2c0-4306-b873-2373a17f1fde admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:01.692765 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046d5c27-f2c0-4306-b873-2373a17f1fde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:01.694531 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046d5c27-f2c0-4306-b873-2373a17f1fde admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:01.718192 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1178/2337] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:16:01 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:16:01.719800 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73c85c3-911e-45c7-800f-c442c4520735 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:01.734770 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73c85c3-911e-45c7-800f-c442c4520735 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:01.741280 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcf42f4-f751-456e-a8a2-352fa924f2dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:01.741942 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcf42f4-f751-456e-a8a2-352fa924f2dd admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:01.742220 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcf42f4-f751-456e-a8a2-352fa924f2dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:01.742625 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcf42f4-f751-456e-a8a2-352fa924f2dd admin admin] PATH_INFO: `/v3/projects/83258ee6a3a647c7aaf4eb729707ca1a/users/153a8689cf5644bf89bff141bab91d19/roles/3f2ca33902a04b6dbc2ef7d06d7289d3` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:01.760512 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcf42f4-f751-456e-a8a2-352fa924f2dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=83258ee6a3a647c7aaf4eb729707ca1a, user_id=153a8689cf5644bf89bff141bab91d19, role_id=3f2ca33902a04b6dbc2ef7d06d7289d3)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:01.764312 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcf42f4-f751-456e-a8a2-352fa924f2dd admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:01.789081 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1160/2338] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:16:01 2022] PUT /identity/v3/projects/83258ee6a3a647c7aaf4eb729707ca1a/users/153a8689cf5644bf89bff141bab91d19/roles/3f2ca33902a04b6dbc2ef7d06d7289d3 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:16:01.793061 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046d5c27-f2c0-4306-b873-2373a17f1fde admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:01.808261 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046d5c27-f2c0-4306-b873-2373a17f1fde admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:01.883939 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4fc95c-bf44-432d-9ba9-3714506d272a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:01.884757 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4fc95c-bf44-432d-9ba9-3714506d272a admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:01.885038 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4fc95c-bf44-432d-9ba9-3714506d272a admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:01.885322 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4fc95c-bf44-432d-9ba9-3714506d272a admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:01.885934 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4fc95c-bf44-432d-9ba9-3714506d272a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:01.887973 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4fc95c-bf44-432d-9ba9-3714506d272a admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:01.901176 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1179/2339] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:16:01 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:16:01.905252 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcf42f4-f751-456e-a8a2-352fa924f2dd admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:01.922500 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcf42f4-f751-456e-a8a2-352fa924f2dd admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:01.950982 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c4dfc8-47e1-4a24-97be-6eb1d080fa97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:01.951722 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c4dfc8-47e1-4a24-97be-6eb1d080fa97 admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:01.952005 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c4dfc8-47e1-4a24-97be-6eb1d080fa97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:01.952283 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c4dfc8-47e1-4a24-97be-6eb1d080fa97 admin admin] PATH_INFO: `/v3/projects/83258ee6a3a647c7aaf4eb729707ca1a/users/153a8689cf5644bf89bff141bab91d19/roles/3f2ca33902a04b6dbc2ef7d06d7289d3` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:01.954413 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c4dfc8-47e1-4a24-97be-6eb1d080fa97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=83258ee6a3a647c7aaf4eb729707ca1a, user_id=153a8689cf5644bf89bff141bab91d19, role_id=3f2ca33902a04b6dbc2ef7d06d7289d3)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:01.960697 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c4dfc8-47e1-4a24-97be-6eb1d080fa97 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:01.978294 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1161/2340] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:16:01 2022] PUT /identity/v3/projects/83258ee6a3a647c7aaf4eb729707ca1a/users/153a8689cf5644bf89bff141bab91d19/roles/3f2ca33902a04b6dbc2ef7d06d7289d3 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:16:02.492749 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4fc95c-bf44-432d-9ba9-3714506d272a admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:02.494815 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-3c4fc95c-bf44-432d-9ba9-3714506d272a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:02.656579 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4fc95c-bf44-432d-9ba9-3714506d272a admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:02.671875 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:16:02.789136 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b83237-3d97-4ed2-8f2e-b640ec649b61 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3adb2f0583c4ff782049dacdb3ee556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:02.790152 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b83237-3d97-4ed2-8f2e-b640ec649b61 service placement] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:02.791953 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b83237-3d97-4ed2-8f2e-b640ec649b61 service placement] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:02.792470 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b83237-3d97-4ed2-8f2e-b640ec649b61 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:02.798655 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-63b83237-3d97-4ed2-8f2e-b640ec649b61 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:02.822969 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b83237-3d97-4ed2-8f2e-b640ec649b61 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:02.824738 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b83237-3d97-4ed2-8f2e-b640ec649b61 service placement] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:02.837416 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1180/2341] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:16:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:16:04.559887 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f0ad15-8c15-4596-a945-2c9f637e5098 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:04.560882 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f0ad15-8c15-4596-a945-2c9f637e5098 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:04.561389 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f0ad15-8c15-4596-a945-2c9f637e5098 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:04.563023 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1162/2342] 10.222.0.49 () {54 vars in 921 bytes} [Fri Sep 9 12:16:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:16:04.576783 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c65519-7160-481f-b6ed-89721ee99535 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:04.577252 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c65519-7160-481f-b6ed-89721ee99535 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:04.577519 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c65519-7160-481f-b6ed-89721ee99535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:04.658315 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-12c65519-7160-481f-b6ed-89721ee99535 None None] MFA Rules not processed for user `70bac169d5cd43e1926f310344ac11fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:16:04.660451 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-12c65519-7160-481f-b6ed-89721ee99535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:04.849775 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1181/2343] 10.222.0.49 () {58 vars in 1018 bytes} [Fri Sep 9 12:16:04 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:16:04.875617 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f0ad15-8c15-4596-a945-2c9f637e5098 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:04.877538 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-f9f0ad15-8c15-4596-a945-2c9f637e5098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:04.968873 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f0ad15-8c15-4596-a945-2c9f637e5098 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:04.974189 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff95fb1a-d6ad-4d62-bd30-f94f1bc1578b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37c4e14aa8e04d3e85a406b9363bca2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:04.974968 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff95fb1a-d6ad-4d62-bd30-f94f1bc1578b service glance] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:04.975338 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff95fb1a-d6ad-4d62-bd30-f94f1bc1578b service glance] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:04.975833 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff95fb1a-d6ad-4d62-bd30-f94f1bc1578b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:04.987784 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff95fb1a-d6ad-4d62-bd30-f94f1bc1578b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:04.996726 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff95fb1a-d6ad-4d62-bd30-f94f1bc1578b service glance] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:05.011597 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1163/2344] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:16:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:16:05.111568 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b627377c-a343-4938-97d9-61a119a1eb48 None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:05.111857 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b627377c-a343-4938-97d9-61a119a1eb48 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:05.112307 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b627377c-a343-4938-97d9-61a119a1eb48 None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:05.114405 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1182/2345] 10.222.0.49 () {54 vars in 921 bytes} [Fri Sep 9 12:16:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:16:05.120165 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a4c6f6-fe98-429a-84e5-9fbee1aaff7b None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:05.121124 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a4c6f6-fe98-429a-84e5-9fbee1aaff7b None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:05.121768 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a4c6f6-fe98-429a-84e5-9fbee1aaff7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:05.148641 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-a1a4c6f6-fe98-429a-84e5-9fbee1aaff7b None None] MFA Rules not processed for user `992de6f86b42495dafe73cb2c393fb4e`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:16:05.150185 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-a1a4c6f6-fe98-429a-84e5-9fbee1aaff7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:05.212057 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1164/2346] 10.222.0.49 () {58 vars in 1018 bytes} [Fri Sep 9 12:16:05 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:16:05.226658 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b627377c-a343-4938-97d9-61a119a1eb48 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:05.238534 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b627377c-a343-4938-97d9-61a119a1eb48 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:05.242408 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd79552-ea7b-4f18-b226-22ebb558740d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:05.243055 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd79552-ea7b-4f18-b226-22ebb558740d service neutron] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:05.243361 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd79552-ea7b-4f18-b226-22ebb558740d service neutron] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:05.243632 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd79552-ea7b-4f18-b226-22ebb558740d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:05.252277 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd79552-ea7b-4f18-b226-22ebb558740d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:05.254291 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd79552-ea7b-4f18-b226-22ebb558740d service neutron] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:05.263185 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1183/2347] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:16:05 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:16:05.373568 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691714a8-c453-445c-9ab5-f1a6ffbfd836 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:05.373862 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691714a8-c453-445c-9ab5-f1a6ffbfd836 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:05.374189 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691714a8-c453-445c-9ab5-f1a6ffbfd836 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:05.375662 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1165/2348] 10.222.0.49 () {54 vars in 921 bytes} [Fri Sep 9 12:16:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:16:05.381928 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c005ce53-3965-4608-8579-8a96942e2318 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:05.382261 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c005ce53-3965-4608-8579-8a96942e2318 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:05.382543 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c005ce53-3965-4608-8579-8a96942e2318 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:05.404048 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-c005ce53-3965-4608-8579-8a96942e2318 None None] MFA Rules not processed for user `70bac169d5cd43e1926f310344ac11fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:16:05.405279 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-c005ce53-3965-4608-8579-8a96942e2318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:05.415505 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1184/2349] 10.222.0.49 () {58 vars in 1018 bytes} [Fri Sep 9 12:16:05 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:16:05.429567 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691714a8-c453-445c-9ab5-f1a6ffbfd836 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:05.442615 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691714a8-c453-445c-9ab5-f1a6ffbfd836 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:05.448628 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd7e552-bd81-4579-b072-6c9122be5619 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:05.449246 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd7e552-bd81-4579-b072-6c9122be5619 service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:05.449522 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd7e552-bd81-4579-b072-6c9122be5619 service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:05.449790 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd7e552-bd81-4579-b072-6c9122be5619 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:05.464857 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd7e552-bd81-4579-b072-6c9122be5619 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:05.466852 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd7e552-bd81-4579-b072-6c9122be5619 service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:05.490268 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1166/2350] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:16:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:16:07.978212 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658b1cbf-b44c-4c3e-a195-f194ee1781fd None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:07.978674 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658b1cbf-b44c-4c3e-a195-f194ee1781fd None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:07.978916 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658b1cbf-b44c-4c3e-a195-f194ee1781fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:08.008235 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-658b1cbf-b44c-4c3e-a195-f194ee1781fd None None] MFA Rules not processed for user `153a8689cf5644bf89bff141bab91d19`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:16:08.009554 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-658b1cbf-b44c-4c3e-a195-f194ee1781fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:08.071345 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:16:08.085010 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1185/2351] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:16:07 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:16:08.099754 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd7e552-bd81-4579-b072-6c9122be5619 service neutron] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:08.102369 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-9bd7e552-bd81-4579-b072-6c9122be5619 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:08.115433 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd7e552-bd81-4579-b072-6c9122be5619 service neutron] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:08.119786 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810688de-c4da-4130-bd5a-4adb4014b82e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37c4e14aa8e04d3e85a406b9363bca2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:08.120589 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810688de-c4da-4130-bd5a-4adb4014b82e service glance] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:08.121018 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810688de-c4da-4130-bd5a-4adb4014b82e service glance] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:08.121579 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810688de-c4da-4130-bd5a-4adb4014b82e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:08.154332 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:16:08.191308 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810688de-c4da-4130-bd5a-4adb4014b82e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:08.192861 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810688de-c4da-4130-bd5a-4adb4014b82e service glance] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:08.202495 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1167/2352] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:16:08 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:16:08.389587 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06b8902-65ae-4c4a-be6c-788c5c3e12c3 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:08.390059 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06b8902-65ae-4c4a-be6c-788c5c3e12c3 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:08.390529 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06b8902-65ae-4c4a-be6c-788c5c3e12c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:08.416972 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-e06b8902-65ae-4c4a-be6c-788c5c3e12c3 None None] MFA Rules not processed for user `e368b4e668c54e25bff80a8e8a8c3588`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:16:08.418608 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-e06b8902-65ae-4c4a-be6c-788c5c3e12c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:08.513899 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1186/2353] 10.222.0.49 () {58 vars in 1009 bytes} [Fri Sep 9 12:16:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:16:08.514972 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810688de-c4da-4130-bd5a-4adb4014b82e service glance] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:08.517037 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-810688de-c4da-4130-bd5a-4adb4014b82e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:08.694559 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810688de-c4da-4130-bd5a-4adb4014b82e service glance] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:08.700773 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e69f61-6788-4d10-9211-16bda89e0d67 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e652ba827849b5a3fc1213d04a1b76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:08.701717 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e69f61-6788-4d10-9211-16bda89e0d67 service swift] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:08.702161 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e69f61-6788-4d10-9211-16bda89e0d67 service swift] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:08.702777 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e69f61-6788-4d10-9211-16bda89e0d67 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:08.729356 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e69f61-6788-4d10-9211-16bda89e0d67 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:08.731017 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e69f61-6788-4d10-9211-16bda89e0d67 service swift] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:08.750370 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1168/2354] 10.222.0.49 () {60 vars in 1384 bytes} [Fri Sep 9 12:16:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 09 12:16:16.830403 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2175e3d-baff-4bbe-926f-c6d063b6f315 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:16.831553 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2175e3d-baff-4bbe-926f-c6d063b6f315 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:16.831553 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2175e3d-baff-4bbe-926f-c6d063b6f315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:16.855514 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-c2175e3d-baff-4bbe-926f-c6d063b6f315 None None] MFA Rules not processed for user `e368b4e668c54e25bff80a8e8a8c3588`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:16:16.856750 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-c2175e3d-baff-4bbe-926f-c6d063b6f315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:16.864181 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1187/2355] 10.222.0.49 () {58 vars in 1009 bytes} [Fri Sep 9 12:16:16 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:16:16.880141 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e69f61-6788-4d10-9211-16bda89e0d67 service swift] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:16.894613 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e69f61-6788-4d10-9211-16bda89e0d67 service swift] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:16.899879 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967996b2-86e2-4721-9d9a-65739aedc08e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e652ba827849b5a3fc1213d04a1b76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:16.900661 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967996b2-86e2-4721-9d9a-65739aedc08e service swift] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:16.900943 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967996b2-86e2-4721-9d9a-65739aedc08e service swift] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:16.901242 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967996b2-86e2-4721-9d9a-65739aedc08e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:16.910533 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967996b2-86e2-4721-9d9a-65739aedc08e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:16.912524 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967996b2-86e2-4721-9d9a-65739aedc08e service swift] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:16.921714 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1169/2356] 10.222.0.49 () {60 vars in 1384 bytes} [Fri Sep 9 12:16:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 12:16:18.831337 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6004342-974f-4fbb-8741-6801e536ddbf None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:18.832162 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6004342-974f-4fbb-8741-6801e536ddbf None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:18.832330 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6004342-974f-4fbb-8741-6801e536ddbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:18.853027 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-f6004342-974f-4fbb-8741-6801e536ddbf None None] MFA Rules not processed for user `eb42a8cde3044a4b9df5cb81d82d9910`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:16:18.854355 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-f6004342-974f-4fbb-8741-6801e536ddbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:18.867441 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1188/2357] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:16:18 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:16:26.918277 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=51 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:16:28.847295 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=34 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:16:29.349814 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfa96f6-8ffd-4ea2-9d01-d1bf96cd0caf None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:29.350353 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfa96f6-8ffd-4ea2-9d01-d1bf96cd0caf None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:29.351181 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfa96f6-8ffd-4ea2-9d01-d1bf96cd0caf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:29.371060 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-cdfa96f6-8ffd-4ea2-9d01-d1bf96cd0caf None None] MFA Rules not processed for user `e368b4e668c54e25bff80a8e8a8c3588`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:16:29.372262 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-cdfa96f6-8ffd-4ea2-9d01-d1bf96cd0caf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:29.386543 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1170/2358] 10.222.0.49 () {58 vars in 1009 bytes} [Fri Sep 9 12:16:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:16:29.398647 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6004342-974f-4fbb-8741-6801e536ddbf None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:29.411275 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6004342-974f-4fbb-8741-6801e536ddbf None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:29.416242 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9693def-c366-4f06-8e17-a6aaf0935d9d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e652ba827849b5a3fc1213d04a1b76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:29.416872 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9693def-c366-4f06-8e17-a6aaf0935d9d service swift] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:29.417150 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9693def-c366-4f06-8e17-a6aaf0935d9d service swift] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:29.417420 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9693def-c366-4f06-8e17-a6aaf0935d9d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:29.426324 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9693def-c366-4f06-8e17-a6aaf0935d9d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:29.427995 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9693def-c366-4f06-8e17-a6aaf0935d9d service swift] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:29.436997 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1189/2359] 10.222.0.49 () {60 vars in 1384 bytes} [Fri Sep 9 12:16:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 12:16:29.639490 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfa96f6-8ffd-4ea2-9d01-d1bf96cd0caf None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:29.640915 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-cdfa96f6-8ffd-4ea2-9d01-d1bf96cd0caf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:29.787518 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfa96f6-8ffd-4ea2-9d01-d1bf96cd0caf None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:29.817825 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74190070-ad3c-4ab6-a74e-706ffd92141b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:29.819515 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74190070-ad3c-4ab6-a74e-706ffd92141b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:29.819799 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74190070-ad3c-4ab6-a74e-706ffd92141b admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:29.820957 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74190070-ad3c-4ab6-a74e-706ffd92141b admin admin] PATH_INFO: `/v3/users/bf11352fb94e4762865702ddd3dc73fe` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:29.824438 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74190070-ad3c-4ab6-a74e-706ffd92141b admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf11352fb94e4762865702ddd3dc73fe)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:29.834805 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74190070-ad3c-4ab6-a74e-706ffd92141b admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:29.957776 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-74190070-ad3c-4ab6-a74e-706ffd92141b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf11352fb94e4762865702ddd3dc73fe'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:29.997311 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-74190070-ad3c-4ab6-a74e-706ffd92141b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf11352fb94e4762865702ddd3dc73fe'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:30.014200 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-74190070-ad3c-4ab6-a74e-706ffd92141b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf11352fb94e4762865702ddd3dc73fe'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:30.037536 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-74190070-ad3c-4ab6-a74e-706ffd92141b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf11352fb94e4762865702ddd3dc73fe'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:30.038106 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-74190070-ad3c-4ab6-a74e-706ffd92141b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf11352fb94e4762865702ddd3dc73fe'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:30.060091 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1171/2360] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 9 12:16:29 2022] DELETE /identity/v3/users/bf11352fb94e4762865702ddd3dc73fe => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:16:30.064541 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62641d7-1430-4e2b-b5ed-de87db2a7a19 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:30.065834 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62641d7-1430-4e2b-b5ed-de87db2a7a19 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:30.065834 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62641d7-1430-4e2b-b5ed-de87db2a7a19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:30.120644 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-d62641d7-1430-4e2b-b5ed-de87db2a7a19 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:16:30.121916 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-d62641d7-1430-4e2b-b5ed-de87db2a7a19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:30.297041 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1190/2361] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:16:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:16:30.314165 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74190070-ad3c-4ab6-a74e-706ffd92141b admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:30.317141 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-74190070-ad3c-4ab6-a74e-706ffd92141b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:30.452981 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74190070-ad3c-4ab6-a74e-706ffd92141b admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:30.457148 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3498ed5-e41e-4b7b-9ab9-8b241cc10dbd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:30.457856 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3498ed5-e41e-4b7b-9ab9-8b241cc10dbd service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:30.458127 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3498ed5-e41e-4b7b-9ab9-8b241cc10dbd service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:30.458400 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3498ed5-e41e-4b7b-9ab9-8b241cc10dbd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:30.472979 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3498ed5-e41e-4b7b-9ab9-8b241cc10dbd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:30.474510 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3498ed5-e41e-4b7b-9ab9-8b241cc10dbd service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:30.488341 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1172/2362] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:16:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:16:30.964118 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62641d7-1430-4e2b-b5ed-de87db2a7a19 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:30.966036 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-d62641d7-1430-4e2b-b5ed-de87db2a7a19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:30.988478 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62641d7-1430-4e2b-b5ed-de87db2a7a19 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:31.007057 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7648209-21fd-44be-b87e-7e0ce32483f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:31.007711 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7648209-21fd-44be-b87e-7e0ce32483f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:31.007986 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7648209-21fd-44be-b87e-7e0ce32483f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:31.008257 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7648209-21fd-44be-b87e-7e0ce32483f8 admin admin] PATH_INFO: `/v3/projects/b37dde899da24a69b858c42c4f499b16` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:31.009403 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7648209-21fd-44be-b87e-7e0ce32483f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b37dde899da24a69b858c42c4f499b16)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:31.011905 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7648209-21fd-44be-b87e-7e0ce32483f8 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:31.139325 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-a7648209-21fd-44be-b87e-7e0ce32483f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b37dde899da24a69b858c42c4f499b16', 'request_id': 'req-a7648209-21fd-44be-b87e-7e0ce32483f8'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:31.148799 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-a7648209-21fd-44be-b87e-7e0ce32483f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b37dde899da24a69b858c42c4f499b16', 'request_id': 'req-a7648209-21fd-44be-b87e-7e0ce32483f8'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:31.155685 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-a7648209-21fd-44be-b87e-7e0ce32483f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b37dde899da24a69b858c42c4f499b16', 'request_id': 'req-a7648209-21fd-44be-b87e-7e0ce32483f8'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:31.186930 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-a7648209-21fd-44be-b87e-7e0ce32483f8 admin admin] The token cache is being invalidate because project b37dde899da24a69b858c42c4f499b16 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81591) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:16:31.187577 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-a7648209-21fd-44be-b87e-7e0ce32483f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:31.188576 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-a7648209-21fd-44be-b87e-7e0ce32483f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:16:31.199676 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1191/2363] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 9 12:16:30 2022] DELETE /identity/v3/projects/b37dde899da24a69b858c42c4f499b16 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:16:40.481678 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=82,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:16:41.177449 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=60,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:16:49.436146 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0350f6-e5b0-49a6-bba0-54c369f92e84 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:49.436681 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0350f6-e5b0-49a6-bba0-54c369f92e84 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:49.436681 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0350f6-e5b0-49a6-bba0-54c369f92e84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:49.484510 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-ca0350f6-e5b0-49a6-bba0-54c369f92e84 None None] MFA Rules not processed for user `e368b4e668c54e25bff80a8e8a8c3588`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:16:49.486077 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-ca0350f6-e5b0-49a6-bba0-54c369f92e84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:49.549019 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1173/2364] 10.222.0.49 () {58 vars in 967 bytes} [Fri Sep 9 12:16:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:16:49.561850 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7648209-21fd-44be-b87e-7e0ce32483f8 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:16:49.563925 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-a7648209-21fd-44be-b87e-7e0ce32483f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:49.635428 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7648209-21fd-44be-b87e-7e0ce32483f8 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:16:49.640428 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859b4370-034f-4e22-828d-468c016479af service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e652ba827849b5a3fc1213d04a1b76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:16:49.641029 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859b4370-034f-4e22-828d-468c016479af service swift] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:49.641303 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859b4370-034f-4e22-828d-468c016479af service swift] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:49.641676 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859b4370-034f-4e22-828d-468c016479af service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:49.650272 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859b4370-034f-4e22-828d-468c016479af service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:16:49.651912 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859b4370-034f-4e22-828d-468c016479af service swift] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:16:49.660507 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1192/2365] 10.222.0.49 () {60 vars in 1384 bytes} [Fri Sep 9 12:16:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 12:16:54.946878 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082079e0-8858-48b6-9604-248f29a0889b None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:16:54.947449 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082079e0-8858-48b6-9604-248f29a0889b None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:16:54.947551 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082079e0-8858-48b6-9604-248f29a0889b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:16:54.968127 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-082079e0-8858-48b6-9604-248f29a0889b None None] MFA Rules not processed for user `2e125f66d9c84e9797099065351a07da`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:16:54.969565 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-082079e0-8858-48b6-9604-248f29a0889b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:16:55.029703 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1174/2366] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:16:54 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:16:59.656688 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=32 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:17:05.022510 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=50 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:17:06.099882 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859b4370-034f-4e22-828d-468c016479af service swift] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:17:06.101395 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-859b4370-034f-4e22-828d-468c016479af service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:17:06.158621 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859b4370-034f-4e22-828d-468c016479af service swift] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:17:06.162233 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a764fa-43da-4fc3-8d0f-fe381fdb3746 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:17:06.162835 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a764fa-43da-4fc3-8d0f-fe381fdb3746 service neutron] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:17:06.163144 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a764fa-43da-4fc3-8d0f-fe381fdb3746 service neutron] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:17:06.163420 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a764fa-43da-4fc3-8d0f-fe381fdb3746 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:17:06.165165 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-27a764fa-43da-4fc3-8d0f-fe381fdb3746 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:17:06.231172 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a764fa-43da-4fc3-8d0f-fe381fdb3746 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:17:06.232612 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a764fa-43da-4fc3-8d0f-fe381fdb3746 service neutron] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:17:06.242439 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1193/2367] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:17:06 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:17:16.238233 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=50 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:17:19.412778 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277720c9-853d-41eb-998b-efd31bdacaba None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:17:19.413332 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277720c9-853d-41eb-998b-efd31bdacaba None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:17:19.413332 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277720c9-853d-41eb-998b-efd31bdacaba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:17:19.432708 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-277720c9-853d-41eb-998b-efd31bdacaba None None] MFA Rules not processed for user `31411178e5c342b7bb14a21dec62f540`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:17:19.433943 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-277720c9-853d-41eb-998b-efd31bdacaba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:17:19.488812 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1175/2368] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:17:19 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:17:29.481957 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=25 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:17:39.631117 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347a5b5e-f36f-4b53-8704-6117870fa574 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:17:39.631636 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347a5b5e-f36f-4b53-8704-6117870fa574 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:17:39.631818 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347a5b5e-f36f-4b53-8704-6117870fa574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:17:39.654948 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-347a5b5e-f36f-4b53-8704-6117870fa574 None None] MFA Rules not processed for user `b9a5c9fb3f664ae097f0c3ce6f6c0764`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:17:39.656157 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-347a5b5e-f36f-4b53-8704-6117870fa574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:17:39.709912 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1194/2369] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:17:39 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:17:42.733041 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277720c9-853d-41eb-998b-efd31bdacaba None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:17:42.743680 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277720c9-853d-41eb-998b-efd31bdacaba None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:17:42.747387 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208acdda-b2a7-4042-badc-5deab68e1755 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:17:42.748680 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208acdda-b2a7-4042-badc-5deab68e1755 service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:17:42.748961 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208acdda-b2a7-4042-badc-5deab68e1755 service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:17:42.749233 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208acdda-b2a7-4042-badc-5deab68e1755 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:17:42.751073 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-208acdda-b2a7-4042-badc-5deab68e1755 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:17:42.827144 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208acdda-b2a7-4042-badc-5deab68e1755 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:17:42.828590 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208acdda-b2a7-4042-badc-5deab68e1755 service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:17:42.840504 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1176/2370] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:17:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:17:43.757722 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347a5b5e-f36f-4b53-8704-6117870fa574 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:17:43.768694 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347a5b5e-f36f-4b53-8704-6117870fa574 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:17:43.772234 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb07ce15-1ec5-452f-8b02-8dbf706a62e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:17:43.772827 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb07ce15-1ec5-452f-8b02-8dbf706a62e4 service neutron] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:17:43.773103 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb07ce15-1ec5-452f-8b02-8dbf706a62e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:17:43.773371 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb07ce15-1ec5-452f-8b02-8dbf706a62e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:17:43.775205 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-bb07ce15-1ec5-452f-8b02-8dbf706a62e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:17:43.788452 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb07ce15-1ec5-452f-8b02-8dbf706a62e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:17:43.789859 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb07ce15-1ec5-452f-8b02-8dbf706a62e4 service neutron] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:17:43.800749 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1195/2371] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:17:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:17:52.835678 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=34 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:17:53.795989 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=31 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:17:53.998312 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208acdda-b2a7-4042-badc-5deab68e1755 service neutron] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:17:53.999867 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-208acdda-b2a7-4042-badc-5deab68e1755 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:17:54.059120 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208acdda-b2a7-4042-badc-5deab68e1755 service neutron] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:17:54.062820 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b336a6f-f04f-4271-929b-a089314381d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535c258100df40b58a5ee44b2c21d621', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:17:54.063577 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b336a6f-f04f-4271-929b-a089314381d3 service cinder] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:17:54.064003 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b336a6f-f04f-4271-929b-a089314381d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:17:54.064320 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b336a6f-f04f-4271-929b-a089314381d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:17:54.066186 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-1b336a6f-f04f-4271-929b-a089314381d3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:17:54.078222 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b336a6f-f04f-4271-929b-a089314381d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:17:54.079687 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b336a6f-f04f-4271-929b-a089314381d3 service cinder] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:17:54.088802 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1177/2372] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:17:53 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:18:02.146571 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb07ce15-1ec5-452f-8b02-8dbf706a62e4 service neutron] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:02.148114 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-bb07ce15-1ec5-452f-8b02-8dbf706a62e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:18:02.159942 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb07ce15-1ec5-452f-8b02-8dbf706a62e4 service neutron] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:02.164292 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebe0da9-a586-4b38-a294-2e1abc6c169f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535c258100df40b58a5ee44b2c21d621', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:02.164894 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebe0da9-a586-4b38-a294-2e1abc6c169f service cinder] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:02.165166 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebe0da9-a586-4b38-a294-2e1abc6c169f service cinder] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:02.165432 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebe0da9-a586-4b38-a294-2e1abc6c169f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:02.167149 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-9ebe0da9-a586-4b38-a294-2e1abc6c169f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:18:02.179871 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebe0da9-a586-4b38-a294-2e1abc6c169f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:02.181329 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebe0da9-a586-4b38-a294-2e1abc6c169f service cinder] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:02.191620 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1196/2373] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:18:02 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:18:04.084558 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=26 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:18:12.187361 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=6 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:18:13.274208 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c946605-eae6-412c-91c3-1bafdd9802c5 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:13.274707 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c946605-eae6-412c-91c3-1bafdd9802c5 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:13.274804 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c946605-eae6-412c-91c3-1bafdd9802c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:13.294457 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-0c946605-eae6-412c-91c3-1bafdd9802c5 None None] MFA Rules not processed for user `2e125f66d9c84e9797099065351a07da`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:18:13.295689 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-0c946605-eae6-412c-91c3-1bafdd9802c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:18:13.302436 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1178/2374] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:18:13 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:18:13.611117 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebe0da9-a586-4b38-a294-2e1abc6c169f service cinder] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:13.620254 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebe0da9-a586-4b38-a294-2e1abc6c169f service cinder] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:13.624594 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d479d59-6247-4ad8-bdeb-38a08c62312f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:13.625171 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d479d59-6247-4ad8-bdeb-38a08c62312f service neutron] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:13.625446 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d479d59-6247-4ad8-bdeb-38a08c62312f service neutron] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:13.625716 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d479d59-6247-4ad8-bdeb-38a08c62312f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:13.627473 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-2d479d59-6247-4ad8-bdeb-38a08c62312f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:18:13.637935 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d479d59-6247-4ad8-bdeb-38a08c62312f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:13.639400 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d479d59-6247-4ad8-bdeb-38a08c62312f service neutron] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:13.647707 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1197/2375] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:18:13 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:18:13.988299 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c946605-eae6-412c-91c3-1bafdd9802c5 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:13.989756 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-0c946605-eae6-412c-91c3-1bafdd9802c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:18:14.070995 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c946605-eae6-412c-91c3-1bafdd9802c5 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:14.076159 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e546fc-2094-4c02-b45b-3e46cb5e9be1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3adb2f0583c4ff782049dacdb3ee556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:14.076772 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e546fc-2094-4c02-b45b-3e46cb5e9be1 service placement] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:14.077051 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e546fc-2094-4c02-b45b-3e46cb5e9be1 service placement] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:14.077324 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e546fc-2094-4c02-b45b-3e46cb5e9be1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:14.079140 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-59e546fc-2094-4c02-b45b-3e46cb5e9be1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:18:14.090375 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e546fc-2094-4c02-b45b-3e46cb5e9be1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:14.091991 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e546fc-2094-4c02-b45b-3e46cb5e9be1 service placement] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:14.102796 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1179/2376] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:18:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:18:17.077987 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d479d59-6247-4ad8-bdeb-38a08c62312f service neutron] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:17.079518 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-2d479d59-6247-4ad8-bdeb-38a08c62312f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:18:17.095574 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d479d59-6247-4ad8-bdeb-38a08c62312f service neutron] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:17.100974 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37978e8e-7b97-4bda-bf5a-51de8c314fdb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70bac169d5cd43e1926f310344ac11fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:17.101570 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37978e8e-7b97-4bda-bf5a-51de8c314fdb service nova] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:17.101903 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37978e8e-7b97-4bda-bf5a-51de8c314fdb service nova] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:17.102175 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37978e8e-7b97-4bda-bf5a-51de8c314fdb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:17.103919 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-37978e8e-7b97-4bda-bf5a-51de8c314fdb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:18:17.116742 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37978e8e-7b97-4bda-bf5a-51de8c314fdb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:17.118307 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37978e8e-7b97-4bda-bf5a-51de8c314fdb service nova] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:17.132377 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1198/2377] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:18:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:18:17.871709 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e546fc-2094-4c02-b45b-3e46cb5e9be1 service placement] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:17.873753 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-59e546fc-2094-4c02-b45b-3e46cb5e9be1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:18:17.890939 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e546fc-2094-4c02-b45b-3e46cb5e9be1 service placement] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:17.896343 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3846cfe3-d397-4f3b-a0c9-ea1970650ece service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:17.896960 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3846cfe3-d397-4f3b-a0c9-ea1970650ece service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:17.897239 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3846cfe3-d397-4f3b-a0c9-ea1970650ece service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:17.897538 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3846cfe3-d397-4f3b-a0c9-ea1970650ece service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:17.899448 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-3846cfe3-d397-4f3b-a0c9-ea1970650ece service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:18:17.917128 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3846cfe3-d397-4f3b-a0c9-ea1970650ece service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:17.918628 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3846cfe3-d397-4f3b-a0c9-ea1970650ece service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:17.928104 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1180/2378] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:18:17 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:18:22.704514 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37978e8e-7b97-4bda-bf5a-51de8c314fdb service nova] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:22.705941 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-37978e8e-7b97-4bda-bf5a-51de8c314fdb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:18:22.722397 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37978e8e-7b97-4bda-bf5a-51de8c314fdb service nova] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:22.727917 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1952980b-714c-4b3e-b0da-600fb5b5a792 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70bac169d5cd43e1926f310344ac11fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:22.728515 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1952980b-714c-4b3e-b0da-600fb5b5a792 service nova] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:22.728792 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1952980b-714c-4b3e-b0da-600fb5b5a792 service nova] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:22.729062 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1952980b-714c-4b3e-b0da-600fb5b5a792 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:22.731563 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-1952980b-714c-4b3e-b0da-600fb5b5a792 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:18:22.743432 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1952980b-714c-4b3e-b0da-600fb5b5a792 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:22.745147 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1952980b-714c-4b3e-b0da-600fb5b5a792 service nova] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:22.758048 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1199/2379] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:18:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:18:25.772400 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3846cfe3-d397-4f3b-a0c9-ea1970650ece service neutron] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:25.782246 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3846cfe3-d397-4f3b-a0c9-ea1970650ece service neutron] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:25.786013 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451cad8b-4970-4758-a27d-3f31b3a9aebb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:25.786678 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451cad8b-4970-4758-a27d-3f31b3a9aebb service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:25.787061 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451cad8b-4970-4758-a27d-3f31b3a9aebb service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:25.787413 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451cad8b-4970-4758-a27d-3f31b3a9aebb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:25.789192 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-451cad8b-4970-4758-a27d-3f31b3a9aebb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:18:25.852191 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451cad8b-4970-4758-a27d-3f31b3a9aebb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:25.853545 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451cad8b-4970-4758-a27d-3f31b3a9aebb service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:25.864109 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1181/2380] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:18:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:18:26.536905 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1952980b-714c-4b3e-b0da-600fb5b5a792 service nova] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:26.552859 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1952980b-714c-4b3e-b0da-600fb5b5a792 service nova] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:26.560836 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451224a7-8a67-4fec-9cf5-411ce6a8a4b1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70bac169d5cd43e1926f310344ac11fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:26.561519 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451224a7-8a67-4fec-9cf5-411ce6a8a4b1 service nova] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:26.562023 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451224a7-8a67-4fec-9cf5-411ce6a8a4b1 service nova] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:26.562357 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451224a7-8a67-4fec-9cf5-411ce6a8a4b1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:26.564841 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-451224a7-8a67-4fec-9cf5-411ce6a8a4b1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:18:26.579188 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451224a7-8a67-4fec-9cf5-411ce6a8a4b1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:26.580891 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451224a7-8a67-4fec-9cf5-411ce6a8a4b1 service nova] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:26.593969 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1200/2381] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:18:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:18:26.845496 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451cad8b-4970-4758-a27d-3f31b3a9aebb service neutron] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:26.855159 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451cad8b-4970-4758-a27d-3f31b3a9aebb service neutron] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:26.858863 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973538bf-3e7f-4279-b0d0-df5a9bdd9a58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:26.859503 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973538bf-3e7f-4279-b0d0-df5a9bdd9a58 service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:26.859781 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973538bf-3e7f-4279-b0d0-df5a9bdd9a58 service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:26.860052 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973538bf-3e7f-4279-b0d0-df5a9bdd9a58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:26.861967 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-973538bf-3e7f-4279-b0d0-df5a9bdd9a58 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:18:26.872573 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973538bf-3e7f-4279-b0d0-df5a9bdd9a58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:26.874022 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973538bf-3e7f-4279-b0d0-df5a9bdd9a58 service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:26.882359 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1182/2382] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:18:26 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:18:27.043988 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd70171-b29a-4475-9ff0-72da7c6c902d None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:27.044531 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd70171-b29a-4475-9ff0-72da7c6c902d None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:27.044898 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd70171-b29a-4475-9ff0-72da7c6c902d None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:27.046477 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1201/2383] 10.222.0.49 () {54 vars in 921 bytes} [Fri Sep 9 12:18:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:18:27.053552 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f7e008-ed46-4db9-b32d-fe12f6a15ae1 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:27.054103 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f7e008-ed46-4db9-b32d-fe12f6a15ae1 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:27.054433 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f7e008-ed46-4db9-b32d-fe12f6a15ae1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:27.075114 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-d3f7e008-ed46-4db9-b32d-fe12f6a15ae1 None None] MFA Rules not processed for user `992de6f86b42495dafe73cb2c393fb4e`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:18:27.076329 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-d3f7e008-ed46-4db9-b32d-fe12f6a15ae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:18:27.083058 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1183/2384] 10.222.0.49 () {58 vars in 1018 bytes} [Fri Sep 9 12:18:27 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:18:27.097040 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd70171-b29a-4475-9ff0-72da7c6c902d None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:27.108853 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd70171-b29a-4475-9ff0-72da7c6c902d None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:27.112550 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741c3701-8340-496a-b5f6-08f4a1152b83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:27.113221 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741c3701-8340-496a-b5f6-08f4a1152b83 service neutron] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:27.113509 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741c3701-8340-496a-b5f6-08f4a1152b83 service neutron] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:27.113780 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741c3701-8340-496a-b5f6-08f4a1152b83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:27.122291 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741c3701-8340-496a-b5f6-08f4a1152b83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:27.123743 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741c3701-8340-496a-b5f6-08f4a1152b83 service neutron] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:27.131948 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1202/2385] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:18:27 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:18:27.222697 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d240a3af-7d01-4184-aa10-e38156d6c9a5 None None] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:27.223128 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d240a3af-7d01-4184-aa10-e38156d6c9a5 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:27.223473 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d240a3af-7d01-4184-aa10-e38156d6c9a5 None None] PATH_INFO: `/` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:27.224922 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1184/2386] 10.222.0.49 () {54 vars in 921 bytes} [Fri Sep 9 12:18:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:18:27.230931 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb7c050-34a8-442d-8a14-0e8fea5e4223 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:27.231232 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb7c050-34a8-442d-8a14-0e8fea5e4223 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:27.231503 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb7c050-34a8-442d-8a14-0e8fea5e4223 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:27.251268 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-2eb7c050-34a8-442d-8a14-0e8fea5e4223 None None] MFA Rules not processed for user `70bac169d5cd43e1926f310344ac11fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:18:27.252471 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-2eb7c050-34a8-442d-8a14-0e8fea5e4223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:18:27.262149 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1203/2387] 10.222.0.49 () {58 vars in 1018 bytes} [Fri Sep 9 12:18:27 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:18:27.303337 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d240a3af-7d01-4184-aa10-e38156d6c9a5 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:27.315205 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d240a3af-7d01-4184-aa10-e38156d6c9a5 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:27.319283 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f3108c-3e4b-4e08-bfef-eda80655ac18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:27.319881 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f3108c-3e4b-4e08-bfef-eda80655ac18 service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:27.320162 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f3108c-3e4b-4e08-bfef-eda80655ac18 service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:27.320490 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f3108c-3e4b-4e08-bfef-eda80655ac18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:27.330994 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f3108c-3e4b-4e08-bfef-eda80655ac18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:27.332437 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f3108c-3e4b-4e08-bfef-eda80655ac18 service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:27.343949 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1185/2388] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:18:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:18:29.720287 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb7c050-34a8-442d-8a14-0e8fea5e4223 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:29.723113 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-2eb7c050-34a8-442d-8a14-0e8fea5e4223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:18:29.740000 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb7c050-34a8-442d-8a14-0e8fea5e4223 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:29.746415 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77640e6b-36ed-43f0-9c97-58886701aa44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:29.747258 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77640e6b-36ed-43f0-9c97-58886701aa44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:29.747542 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77640e6b-36ed-43f0-9c97-58886701aa44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:29.747928 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77640e6b-36ed-43f0-9c97-58886701aa44 admin admin] PATH_INFO: `/v3/users/31411178e5c342b7bb14a21dec62f540` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:29.749639 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77640e6b-36ed-43f0-9c97-58886701aa44 admin admin] RBAC: Authorizing `identity:delete_user(user_id=31411178e5c342b7bb14a21dec62f540)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:29.752053 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77640e6b-36ed-43f0-9c97-58886701aa44 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:29.832503 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-77640e6b-36ed-43f0-9c97-58886701aa44 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31411178e5c342b7bb14a21dec62f540'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:29.840142 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-77640e6b-36ed-43f0-9c97-58886701aa44 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31411178e5c342b7bb14a21dec62f540'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:29.858078 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-77640e6b-36ed-43f0-9c97-58886701aa44 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31411178e5c342b7bb14a21dec62f540'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:29.867554 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-77640e6b-36ed-43f0-9c97-58886701aa44 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31411178e5c342b7bb14a21dec62f540'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:29.868214 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-77640e6b-36ed-43f0-9c97-58886701aa44 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31411178e5c342b7bb14a21dec62f540'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:29.876343 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1204/2389] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 9 12:18:29 2022] DELETE /identity/v3/users/31411178e5c342b7bb14a21dec62f540 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:18:30.096156 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f3108c-3e4b-4e08-bfef-eda80655ac18 service neutron] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:30.110673 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f3108c-3e4b-4e08-bfef-eda80655ac18 service neutron] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:30.149720 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:18:30.183512 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb21a7aa-9170-4c6a-9284-c22234e3e2db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:30.184566 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb21a7aa-9170-4c6a-9284-c22234e3e2db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:30.184849 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb21a7aa-9170-4c6a-9284-c22234e3e2db admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:30.185119 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb21a7aa-9170-4c6a-9284-c22234e3e2db admin admin] PATH_INFO: `/v3/projects/edbc1c04ea364f3bb335fd4ad0731d1b` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:30.186879 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb21a7aa-9170-4c6a-9284-c22234e3e2db admin admin] RBAC: Authorizing `identity:delete_project(project_id=edbc1c04ea364f3bb335fd4ad0731d1b)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:30.189465 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb21a7aa-9170-4c6a-9284-c22234e3e2db admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:30.246700 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-bb21a7aa-9170-4c6a-9284-c22234e3e2db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'edbc1c04ea364f3bb335fd4ad0731d1b', 'request_id': 'req-bb21a7aa-9170-4c6a-9284-c22234e3e2db'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:30.256281 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-bb21a7aa-9170-4c6a-9284-c22234e3e2db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'edbc1c04ea364f3bb335fd4ad0731d1b', 'request_id': 'req-bb21a7aa-9170-4c6a-9284-c22234e3e2db'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:30.265560 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-bb21a7aa-9170-4c6a-9284-c22234e3e2db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'edbc1c04ea364f3bb335fd4ad0731d1b', 'request_id': 'req-bb21a7aa-9170-4c6a-9284-c22234e3e2db'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:30.279491 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-bb21a7aa-9170-4c6a-9284-c22234e3e2db admin admin] The token cache is being invalidate because project edbc1c04ea364f3bb335fd4ad0731d1b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:18:30.279831 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-bb21a7aa-9170-4c6a-9284-c22234e3e2db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:30.280433 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-bb21a7aa-9170-4c6a-9284-c22234e3e2db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:30.282336 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1186/2390] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 9 12:18:30 2022] DELETE /identity/v3/projects/edbc1c04ea364f3bb335fd4ad0731d1b => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:18:30.287621 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ff618e-2b75-40b3-b638-e456bdcd4aa4 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:30.287939 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ff618e-2b75-40b3-b638-e456bdcd4aa4 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:30.288166 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ff618e-2b75-40b3-b638-e456bdcd4aa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:30.314553 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-e2ff618e-2b75-40b3-b638-e456bdcd4aa4 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:18:30.315955 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-e2ff618e-2b75-40b3-b638-e456bdcd4aa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:18:30.379461 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:18:30.392332 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1205/2391] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:18:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:18:30.399930 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87640ffd-cc13-454f-80b7-b230a4438ee1 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:30.400216 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87640ffd-cc13-454f-80b7-b230a4438ee1 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:30.400488 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87640ffd-cc13-454f-80b7-b230a4438ee1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:30.427848 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-87640ffd-cc13-454f-80b7-b230a4438ee1 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:18:30.429102 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-87640ffd-cc13-454f-80b7-b230a4438ee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:18:30.437392 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1187/2392] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:18:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:18:30.441684 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ff618e-2b75-40b3-b638-e456bdcd4aa4 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:30.451790 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ff618e-2b75-40b3-b638-e456bdcd4aa4 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:30.457645 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cfca56-4f33-418d-a771-d3a13af88db2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:30.458576 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cfca56-4f33-418d-a771-d3a13af88db2 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:30.458859 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cfca56-4f33-418d-a771-d3a13af88db2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:30.459161 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cfca56-4f33-418d-a771-d3a13af88db2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:30.459939 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cfca56-4f33-418d-a771-d3a13af88db2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:18:30.460243 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cfca56-4f33-418d-a771-d3a13af88db2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:30.461841 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cfca56-4f33-418d-a771-d3a13af88db2 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:30.474717 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1206/2393] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 9 12:18:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:18:30.481088 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87640ffd-cc13-454f-80b7-b230a4438ee1 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:30.493565 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87640ffd-cc13-454f-80b7-b230a4438ee1 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:30.499023 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7638977a-b8a6-4f53-929b-a13084c12112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:30.499638 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7638977a-b8a6-4f53-929b-a13084c12112 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:30.499916 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7638977a-b8a6-4f53-929b-a13084c12112 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:30.500185 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7638977a-b8a6-4f53-929b-a13084c12112 admin admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:30.500812 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7638977a-b8a6-4f53-929b-a13084c12112 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:30.502715 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7638977a-b8a6-4f53-929b-a13084c12112 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:30.534267 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1188/2394] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 9 12:18:30 2022] POST /identity/v3/projects => generated 403 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:18:30.545686 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cfca56-4f33-418d-a771-d3a13af88db2 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:30.562766 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cfca56-4f33-418d-a771-d3a13af88db2 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:30.663746 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb33088-ab75-4506-b0b6-89df1217aa1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:30.664996 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb33088-ab75-4506-b0b6-89df1217aa1c admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:30.665386 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb33088-ab75-4506-b0b6-89df1217aa1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:30.666515 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb33088-ab75-4506-b0b6-89df1217aa1c admin admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:30.666515 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb33088-ab75-4506-b0b6-89df1217aa1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:30.669822 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb33088-ab75-4506-b0b6-89df1217aa1c admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:30.695721 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1207/2395] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:18:30 2022] POST /identity/v3/users => generated 302 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:18:30.700297 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7638977a-b8a6-4f53-929b-a13084c12112 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:30.711167 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7638977a-b8a6-4f53-929b-a13084c12112 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:30.716420 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93cd4e4-649b-4615-a1ff-ad163d9112ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:30.717031 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93cd4e4-649b-4615-a1ff-ad163d9112ce admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:30.717305 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93cd4e4-649b-4615-a1ff-ad163d9112ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:30.717574 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93cd4e4-649b-4615-a1ff-ad163d9112ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:30.718212 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93cd4e4-649b-4615-a1ff-ad163d9112ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:30.720068 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93cd4e4-649b-4615-a1ff-ad163d9112ce admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:30.730923 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1189/2396] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:18:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:18:30.734791 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb33088-ab75-4506-b0b6-89df1217aa1c admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:30.745069 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb33088-ab75-4506-b0b6-89df1217aa1c admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:30.750253 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d037b7e-260e-46b4-8e37-75295cd9d40a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:30.750990 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d037b7e-260e-46b4-8e37-75295cd9d40a admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:30.751270 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d037b7e-260e-46b4-8e37-75295cd9d40a admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:30.751538 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d037b7e-260e-46b4-8e37-75295cd9d40a admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:30.752205 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d037b7e-260e-46b4-8e37-75295cd9d40a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:30.753802 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d037b7e-260e-46b4-8e37-75295cd9d40a admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:30.765838 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1208/2397] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:18:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:18:30.770180 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93cd4e4-649b-4615-a1ff-ad163d9112ce admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:30.781487 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93cd4e4-649b-4615-a1ff-ad163d9112ce admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:30.786464 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5184da-e3a6-4671-a485-56531a5692cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:30.787242 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5184da-e3a6-4671-a485-56531a5692cb admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:30.787520 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5184da-e3a6-4671-a485-56531a5692cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:30.787838 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5184da-e3a6-4671-a485-56531a5692cb admin admin] PATH_INFO: `/v3/projects/e634a57bba6b439593da971d8ac8b99f/users/af0f1408eaa44ba5999ec113d61608fe/roles/eecd2f160dd844f5906f3d345780c501` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:30.803300 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5184da-e3a6-4671-a485-56531a5692cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=e634a57bba6b439593da971d8ac8b99f, user_id=af0f1408eaa44ba5999ec113d61608fe, role_id=eecd2f160dd844f5906f3d345780c501)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:30.806270 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5184da-e3a6-4671-a485-56531a5692cb admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:30.819849 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1190/2398] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:18:30 2022] PUT /identity/v3/projects/e634a57bba6b439593da971d8ac8b99f/users/af0f1408eaa44ba5999ec113d61608fe/roles/eecd2f160dd844f5906f3d345780c501 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:18:30.825568 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befbd41b-aeeb-4e57-8db3-4fc8ffebe1c4 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:30.825849 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befbd41b-aeeb-4e57-8db3-4fc8ffebe1c4 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:30.826111 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befbd41b-aeeb-4e57-8db3-4fc8ffebe1c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:30.846028 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-befbd41b-aeeb-4e57-8db3-4fc8ffebe1c4 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:18:30.847282 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-befbd41b-aeeb-4e57-8db3-4fc8ffebe1c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:18:30.910769 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1209/2399] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:18:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:18:30.920606 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5184da-e3a6-4671-a485-56531a5692cb admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:30.922373 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-4b5184da-e3a6-4671-a485-56531a5692cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:18:30.981295 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5184da-e3a6-4671-a485-56531a5692cb admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:30.985063 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cb01ea-ac5f-4067-bfba-ae12a82db7a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:30.985668 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cb01ea-ac5f-4067-bfba-ae12a82db7a9 service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:30.985974 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cb01ea-ac5f-4067-bfba-ae12a82db7a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:30.986245 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cb01ea-ac5f-4067-bfba-ae12a82db7a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:30.996400 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cb01ea-ac5f-4067-bfba-ae12a82db7a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:30.997848 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cb01ea-ac5f-4067-bfba-ae12a82db7a9 service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:31.002987 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:18:31.008737 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1191/2400] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:18:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:18:31.992381 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaacb64-2a02-4f20-8184-010ba39479fc None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:31.992900 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaacb64-2a02-4f20-8184-010ba39479fc None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:31.993118 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaacb64-2a02-4f20-8184-010ba39479fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:32.030388 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-fcaacb64-2a02-4f20-8184-010ba39479fc None None] MFA Rules not processed for user `af0f1408eaa44ba5999ec113d61608fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:18:32.032049 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-fcaacb64-2a02-4f20-8184-010ba39479fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:18:32.103066 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:18:32.116061 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1210/2401] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:18:31 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:18:32.117722 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cb01ea-ac5f-4067-bfba-ae12a82db7a9 service neutron] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:32.128521 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cb01ea-ac5f-4067-bfba-ae12a82db7a9 service neutron] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:32.133204 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2868912d-4af6-4140-b7df-44a06f184c6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:32.133824 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2868912d-4af6-4140-b7df-44a06f184c6e admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:32.134100 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2868912d-4af6-4140-b7df-44a06f184c6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:32.134369 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2868912d-4af6-4140-b7df-44a06f184c6e admin admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:32.135579 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2868912d-4af6-4140-b7df-44a06f184c6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:32.137864 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2868912d-4af6-4140-b7df-44a06f184c6e admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:32.177862 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaacb64-2a02-4f20-8184-010ba39479fc None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:32.179950 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1192/2402] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 9 12:18:32 2022] POST /identity/v3/projects => generated 405 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:18:32.192112 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaacb64-2a02-4f20-8184-010ba39479fc None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:32.269612 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18223a11-4fc2-406e-a9fc-a479162196c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:32.270237 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18223a11-4fc2-406e-a9fc-a479162196c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:32.270516 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18223a11-4fc2-406e-a9fc-a479162196c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:32.270784 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18223a11-4fc2-406e-a9fc-a479162196c8 admin admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:32.271749 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18223a11-4fc2-406e-a9fc-a479162196c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:32.273780 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18223a11-4fc2-406e-a9fc-a479162196c8 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:32.299720 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1211/2403] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:18:32 2022] POST /identity/v3/users => generated 309 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:18:32.303939 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2868912d-4af6-4140-b7df-44a06f184c6e admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:32.315507 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2868912d-4af6-4140-b7df-44a06f184c6e admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:32.320211 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855b4cbe-19e3-408d-add6-e28a93656709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:32.320826 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855b4cbe-19e3-408d-add6-e28a93656709 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:32.321106 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855b4cbe-19e3-408d-add6-e28a93656709 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:32.321377 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855b4cbe-19e3-408d-add6-e28a93656709 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:32.322038 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855b4cbe-19e3-408d-add6-e28a93656709 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:32.325180 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855b4cbe-19e3-408d-add6-e28a93656709 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:32.336242 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1193/2404] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:18:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:18:32.341008 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18223a11-4fc2-406e-a9fc-a479162196c8 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:32.352240 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18223a11-4fc2-406e-a9fc-a479162196c8 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:32.358062 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0660c37-4bdd-4f17-a318-9b26a244fd6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:32.358747 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0660c37-4bdd-4f17-a318-9b26a244fd6b admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:32.359046 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0660c37-4bdd-4f17-a318-9b26a244fd6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:32.359322 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0660c37-4bdd-4f17-a318-9b26a244fd6b admin admin] PATH_INFO: `/v3/projects/1e59fcb1cb6241248e818f77e9c5cc55/users/4efd36c5e1b24e6dbaf89e4e9f7cbf80/roles/3f2ca33902a04b6dbc2ef7d06d7289d3` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:32.377560 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0660c37-4bdd-4f17-a318-9b26a244fd6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e59fcb1cb6241248e818f77e9c5cc55, user_id=4efd36c5e1b24e6dbaf89e4e9f7cbf80, role_id=3f2ca33902a04b6dbc2ef7d06d7289d3)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:32.380411 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0660c37-4bdd-4f17-a318-9b26a244fd6b admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:32.397298 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1212/2405] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:18:32 2022] PUT /identity/v3/projects/1e59fcb1cb6241248e818f77e9c5cc55/users/4efd36c5e1b24e6dbaf89e4e9f7cbf80/roles/3f2ca33902a04b6dbc2ef7d06d7289d3 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:18:32.401398 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855b4cbe-19e3-408d-add6-e28a93656709 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:32.412642 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855b4cbe-19e3-408d-add6-e28a93656709 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:32.487223 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c44c73a-12bd-4d86-9237-5a37478b7971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:32.487989 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c44c73a-12bd-4d86-9237-5a37478b7971 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:32.488271 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c44c73a-12bd-4d86-9237-5a37478b7971 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:32.488546 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c44c73a-12bd-4d86-9237-5a37478b7971 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:32.489530 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c44c73a-12bd-4d86-9237-5a37478b7971 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:32.491725 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c44c73a-12bd-4d86-9237-5a37478b7971 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:32.513866 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1194/2406] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:18:32 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:18:32.514048 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0660c37-4bdd-4f17-a318-9b26a244fd6b admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:32.527761 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0660c37-4bdd-4f17-a318-9b26a244fd6b admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:32.539598 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8fe8e2-da14-484b-9b7e-6aae97f7f6ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:32.540431 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8fe8e2-da14-484b-9b7e-6aae97f7f6ef admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:32.540718 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8fe8e2-da14-484b-9b7e-6aae97f7f6ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:32.540995 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8fe8e2-da14-484b-9b7e-6aae97f7f6ef admin admin] PATH_INFO: `/v3/projects/1e59fcb1cb6241248e818f77e9c5cc55/users/4efd36c5e1b24e6dbaf89e4e9f7cbf80/roles/3f2ca33902a04b6dbc2ef7d06d7289d3` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:32.549534 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8fe8e2-da14-484b-9b7e-6aae97f7f6ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e59fcb1cb6241248e818f77e9c5cc55, user_id=4efd36c5e1b24e6dbaf89e4e9f7cbf80, role_id=3f2ca33902a04b6dbc2ef7d06d7289d3)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:32.552325 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8fe8e2-da14-484b-9b7e-6aae97f7f6ef admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:32.572619 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1213/2407] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:18:32 2022] PUT /identity/v3/projects/1e59fcb1cb6241248e818f77e9c5cc55/users/4efd36c5e1b24e6dbaf89e4e9f7cbf80/roles/3f2ca33902a04b6dbc2ef7d06d7289d3 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:18:33.764291 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8011b9f5-666a-40ba-bfb5-057f2c01fbca None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:33.765191 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8011b9f5-666a-40ba-bfb5-057f2c01fbca None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:33.765686 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8011b9f5-666a-40ba-bfb5-057f2c01fbca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:33.808365 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-8011b9f5-666a-40ba-bfb5-057f2c01fbca None None] MFA Rules not processed for user `4efd36c5e1b24e6dbaf89e4e9f7cbf80`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:18:33.809760 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-8011b9f5-666a-40ba-bfb5-057f2c01fbca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:18:33.983618 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1195/2408] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:18:33 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:18:34.954803 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c6b35d-0c66-46c0-a3f4-87f3b918fa8e None None] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:34.955401 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c6b35d-0c66-46c0-a3f4-87f3b918fa8e None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:34.955401 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c6b35d-0c66-46c0-a3f4-87f3b918fa8e None None] PATH_INFO: `/` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:34.956780 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1214/2409] 10.222.0.49 () {54 vars in 921 bytes} [Fri Sep 9 12:18:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 12:18:34.988357 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9215ac43-97e6-4a28-b223-e3d4cf1ef378 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:34.988865 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9215ac43-97e6-4a28-b223-e3d4cf1ef378 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:34.988949 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9215ac43-97e6-4a28-b223-e3d4cf1ef378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:35.013204 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-9215ac43-97e6-4a28-b223-e3d4cf1ef378 None None] MFA Rules not processed for user `d3adb2f0583c4ff782049dacdb3ee556`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:18:35.014655 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-9215ac43-97e6-4a28-b223-e3d4cf1ef378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:18:35.075455 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:18:35.097449 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1196/2410] 10.222.0.49 () {58 vars in 1018 bytes} [Fri Sep 9 12:18:34 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:18:35.109847 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c6b35d-0c66-46c0-a3f4-87f3b918fa8e None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:35.111730 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-14c6b35d-0c66-46c0-a3f4-87f3b918fa8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:18:35.129035 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c6b35d-0c66-46c0-a3f4-87f3b918fa8e None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:35.134078 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e08be7-3d67-41e8-a14a-ab2794b78869 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3adb2f0583c4ff782049dacdb3ee556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:35.135366 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e08be7-3d67-41e8-a14a-ab2794b78869 service placement] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:35.135699 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e08be7-3d67-41e8-a14a-ab2794b78869 service placement] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:35.135975 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e08be7-3d67-41e8-a14a-ab2794b78869 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:35.144965 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e08be7-3d67-41e8-a14a-ab2794b78869 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:35.146539 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e08be7-3d67-41e8-a14a-ab2794b78869 service placement] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:35.156902 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1215/2411] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:18:35 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:18:35.542276 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9215ac43-97e6-4a28-b223-e3d4cf1ef378 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:35.614913 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9215ac43-97e6-4a28-b223-e3d4cf1ef378 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:35.620109 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b4e3e5-42c4-4a67-b073-03f603c2a984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:35.620777 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b4e3e5-42c4-4a67-b073-03f603c2a984 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:35.621053 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b4e3e5-42c4-4a67-b073-03f603c2a984 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:35.621325 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b4e3e5-42c4-4a67-b073-03f603c2a984 admin admin] PATH_INFO: `/v3/users/af0f1408eaa44ba5999ec113d61608fe` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:35.622439 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b4e3e5-42c4-4a67-b073-03f603c2a984 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af0f1408eaa44ba5999ec113d61608fe)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:35.624391 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b4e3e5-42c4-4a67-b073-03f603c2a984 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:35.670683 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-64b4e3e5-42c4-4a67-b073-03f603c2a984 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af0f1408eaa44ba5999ec113d61608fe'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:35.683484 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-64b4e3e5-42c4-4a67-b073-03f603c2a984 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af0f1408eaa44ba5999ec113d61608fe'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:35.690256 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-64b4e3e5-42c4-4a67-b073-03f603c2a984 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af0f1408eaa44ba5999ec113d61608fe'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:35.697163 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-64b4e3e5-42c4-4a67-b073-03f603c2a984 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af0f1408eaa44ba5999ec113d61608fe'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:35.698635 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-64b4e3e5-42c4-4a67-b073-03f603c2a984 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af0f1408eaa44ba5999ec113d61608fe'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:35.704530 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1197/2412] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 9 12:18:35 2022] DELETE /identity/v3/users/af0f1408eaa44ba5999ec113d61608fe => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:18:35.875021 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e08be7-3d67-41e8-a14a-ab2794b78869 service placement] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:35.885868 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e08be7-3d67-41e8-a14a-ab2794b78869 service placement] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:35.956653 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f14ad5-7126-4666-9998-be6b603449b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:35.957350 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f14ad5-7126-4666-9998-be6b603449b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:35.957595 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f14ad5-7126-4666-9998-be6b603449b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:35.957838 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f14ad5-7126-4666-9998-be6b603449b1 admin admin] PATH_INFO: `/v3/projects/e634a57bba6b439593da971d8ac8b99f` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:35.959654 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f14ad5-7126-4666-9998-be6b603449b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e634a57bba6b439593da971d8ac8b99f)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:35.961607 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f14ad5-7126-4666-9998-be6b603449b1 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:35.980621 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b4e3e5-42c4-4a67-b073-03f603c2a984 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:35.982223 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-64b4e3e5-42c4-4a67-b073-03f603c2a984 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:18:36.006149 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:18:36.019061 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-62f14ad5-7126-4666-9998-be6b603449b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e634a57bba6b439593da971d8ac8b99f', 'request_id': 'req-62f14ad5-7126-4666-9998-be6b603449b1'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:36.026441 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-62f14ad5-7126-4666-9998-be6b603449b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e634a57bba6b439593da971d8ac8b99f', 'request_id': 'req-62f14ad5-7126-4666-9998-be6b603449b1'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:36.033329 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-62f14ad5-7126-4666-9998-be6b603449b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e634a57bba6b439593da971d8ac8b99f', 'request_id': 'req-62f14ad5-7126-4666-9998-be6b603449b1'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:36.050947 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-62f14ad5-7126-4666-9998-be6b603449b1 admin admin] The token cache is being invalidate because project e634a57bba6b439593da971d8ac8b99f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81591) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:18:36.050947 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-62f14ad5-7126-4666-9998-be6b603449b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:36.051968 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-62f14ad5-7126-4666-9998-be6b603449b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:36.055360 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1216/2413] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 9 12:18:35 2022] DELETE /identity/v3/projects/e634a57bba6b439593da971d8ac8b99f => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:18:36.060038 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f14ad5-7126-4666-9998-be6b603449b1 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:36.061571 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-62f14ad5-7126-4666-9998-be6b603449b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:18:36.084862 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:18:36.097005 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b4e3e5-42c4-4a67-b073-03f603c2a984 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:36.137769 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f14ad5-7126-4666-9998-be6b603449b1 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:36.143530 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b9cbc8-b1bc-4351-ae32-445f758958b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:36.144175 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b9cbc8-b1bc-4351-ae32-445f758958b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:36.144457 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b9cbc8-b1bc-4351-ae32-445f758958b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:36.144733 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b9cbc8-b1bc-4351-ae32-445f758958b8 admin admin] PATH_INFO: `/v3/users/4efd36c5e1b24e6dbaf89e4e9f7cbf80` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:36.146766 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b9cbc8-b1bc-4351-ae32-445f758958b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4efd36c5e1b24e6dbaf89e4e9f7cbf80)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:36.148737 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b9cbc8-b1bc-4351-ae32-445f758958b8 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:36.175637 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b67ddd-a84d-40b1-acbb-17843d84fcd8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70bac169d5cd43e1926f310344ac11fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:36.176663 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b67ddd-a84d-40b1-acbb-17843d84fcd8 service nova] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:36.177084 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b67ddd-a84d-40b1-acbb-17843d84fcd8 service nova] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:36.177414 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b67ddd-a84d-40b1-acbb-17843d84fcd8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:36.179896 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-83b67ddd-a84d-40b1-acbb-17843d84fcd8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:18:36.206571 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-51b9cbc8-b1bc-4351-ae32-445f758958b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4efd36c5e1b24e6dbaf89e4e9f7cbf80'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:36.214237 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-51b9cbc8-b1bc-4351-ae32-445f758958b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4efd36c5e1b24e6dbaf89e4e9f7cbf80'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:36.236089 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-51b9cbc8-b1bc-4351-ae32-445f758958b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4efd36c5e1b24e6dbaf89e4e9f7cbf80'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:36.244297 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-51b9cbc8-b1bc-4351-ae32-445f758958b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4efd36c5e1b24e6dbaf89e4e9f7cbf80'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:36.244499 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b67ddd-a84d-40b1-acbb-17843d84fcd8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:36.245372 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-51b9cbc8-b1bc-4351-ae32-445f758958b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4efd36c5e1b24e6dbaf89e4e9f7cbf80'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:36.246441 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b67ddd-a84d-40b1-acbb-17843d84fcd8 service nova] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:36.252269 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1217/2414] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 9 12:18:36 2022] DELETE /identity/v3/users/4efd36c5e1b24e6dbaf89e4e9f7cbf80 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:18:36.258685 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1198/2415] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:18:35 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:18:36.435142 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b9cbc8-b1bc-4351-ae32-445f758958b8 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:36.436854 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-51b9cbc8-b1bc-4351-ae32-445f758958b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:18:36.492303 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:18:36.514126 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b9cbc8-b1bc-4351-ae32-445f758958b8 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:36.520407 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4ffa8c-0673-41d8-8055-938813703630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:36.521129 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4ffa8c-0673-41d8-8055-938813703630 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:36.521406 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4ffa8c-0673-41d8-8055-938813703630 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:36.521678 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4ffa8c-0673-41d8-8055-938813703630 admin admin] PATH_INFO: `/v3/projects/1e59fcb1cb6241248e818f77e9c5cc55` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:36.522934 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4ffa8c-0673-41d8-8055-938813703630 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e59fcb1cb6241248e818f77e9c5cc55)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:36.524928 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4ffa8c-0673-41d8-8055-938813703630 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:36.595053 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-ab4ffa8c-0673-41d8-8055-938813703630 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e59fcb1cb6241248e818f77e9c5cc55', 'request_id': 'req-ab4ffa8c-0673-41d8-8055-938813703630'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:36.600958 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-ab4ffa8c-0673-41d8-8055-938813703630 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e59fcb1cb6241248e818f77e9c5cc55', 'request_id': 'req-ab4ffa8c-0673-41d8-8055-938813703630'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:36.607398 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-ab4ffa8c-0673-41d8-8055-938813703630 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e59fcb1cb6241248e818f77e9c5cc55', 'request_id': 'req-ab4ffa8c-0673-41d8-8055-938813703630'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:36.620853 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-ab4ffa8c-0673-41d8-8055-938813703630 admin admin] The token cache is being invalidate because project 1e59fcb1cb6241248e818f77e9c5cc55 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81591) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:18:36.621082 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-ab4ffa8c-0673-41d8-8055-938813703630 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:36.621781 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-ab4ffa8c-0673-41d8-8055-938813703630 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:36.626914 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1218/2416] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 9 12:18:36 2022] DELETE /identity/v3/projects/1e59fcb1cb6241248e818f77e9c5cc55 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:18:36.628593 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c11b31f-ef9f-48a4-b8f2-5178361ec5c2 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:36.628882 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c11b31f-ef9f-48a4-b8f2-5178361ec5c2 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:36.629178 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c11b31f-ef9f-48a4-b8f2-5178361ec5c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:36.653860 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-4c11b31f-ef9f-48a4-b8f2-5178361ec5c2 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:18:36.655166 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-4c11b31f-ef9f-48a4-b8f2-5178361ec5c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:18:36.720491 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1199/2417] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:18:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:18:36.728717 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c28b183-6b58-45e3-baa2-4075a6b86301 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:36.729005 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c28b183-6b58-45e3-baa2-4075a6b86301 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:36.729278 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c28b183-6b58-45e3-baa2-4075a6b86301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:36.750647 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-0c28b183-6b58-45e3-baa2-4075a6b86301 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:18:36.751928 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-0c28b183-6b58-45e3-baa2-4075a6b86301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:18:36.760298 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1219/2418] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:18:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:18:36.764609 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c11b31f-ef9f-48a4-b8f2-5178361ec5c2 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:36.774563 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c11b31f-ef9f-48a4-b8f2-5178361ec5c2 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:36.779352 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0702330d-cb39-473b-a279-1d1c574b91e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:36.779967 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0702330d-cb39-473b-a279-1d1c574b91e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:36.780245 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0702330d-cb39-473b-a279-1d1c574b91e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:36.780515 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0702330d-cb39-473b-a279-1d1c574b91e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:36.781194 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0702330d-cb39-473b-a279-1d1c574b91e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:18:36.781495 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0702330d-cb39-473b-a279-1d1c574b91e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:36.783121 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0702330d-cb39-473b-a279-1d1c574b91e0 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:36.795950 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1200/2419] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 9 12:18:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:18:36.800808 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c28b183-6b58-45e3-baa2-4075a6b86301 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:36.811485 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c28b183-6b58-45e3-baa2-4075a6b86301 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:36.816117 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d5ba24-27d1-4e02-8200-75140b0b471d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:36.816731 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d5ba24-27d1-4e02-8200-75140b0b471d admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:36.817005 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d5ba24-27d1-4e02-8200-75140b0b471d admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:36.817272 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d5ba24-27d1-4e02-8200-75140b0b471d admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:36.817890 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d5ba24-27d1-4e02-8200-75140b0b471d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:36.819812 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d5ba24-27d1-4e02-8200-75140b0b471d admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:36.846259 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1220/2420] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 9 12:18:36 2022] POST /identity/v3/projects => generated 365 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:18:36.850767 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0702330d-cb39-473b-a279-1d1c574b91e0 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:36.862537 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0702330d-cb39-473b-a279-1d1c574b91e0 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:36.886772 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:18:36.919892 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb0c8f3-e169-4bf4-ae3f-3d9cbb247aed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:36.920518 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb0c8f3-e169-4bf4-ae3f-3d9cbb247aed admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:36.920789 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb0c8f3-e169-4bf4-ae3f-3d9cbb247aed admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:36.921053 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb0c8f3-e169-4bf4-ae3f-3d9cbb247aed admin admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:36.921670 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb0c8f3-e169-4bf4-ae3f-3d9cbb247aed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:36.923632 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb0c8f3-e169-4bf4-ae3f-3d9cbb247aed admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:36.944130 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1201/2421] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:18:36 2022] POST /identity/v3/users => generated 283 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:18:36.948442 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d5ba24-27d1-4e02-8200-75140b0b471d admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:36.958647 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d5ba24-27d1-4e02-8200-75140b0b471d admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:36.964023 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca7db37-97ab-4299-9117-0c46c4df1565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:36.964695 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca7db37-97ab-4299-9117-0c46c4df1565 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:36.964976 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca7db37-97ab-4299-9117-0c46c4df1565 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:36.965248 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca7db37-97ab-4299-9117-0c46c4df1565 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:36.965889 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca7db37-97ab-4299-9117-0c46c4df1565 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:36.967516 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca7db37-97ab-4299-9117-0c46c4df1565 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:36.977873 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1221/2422] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:18:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 12:18:36.982655 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb0c8f3-e169-4bf4-ae3f-3d9cbb247aed admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:36.992533 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb0c8f3-e169-4bf4-ae3f-3d9cbb247aed admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:36.997296 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10e0194-58ec-4ad7-a8eb-bc747cdb93eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:36.997881 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10e0194-58ec-4ad7-a8eb-bc747cdb93eb admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:36.998174 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10e0194-58ec-4ad7-a8eb-bc747cdb93eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:36.998429 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10e0194-58ec-4ad7-a8eb-bc747cdb93eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:36.999093 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10e0194-58ec-4ad7-a8eb-bc747cdb93eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:37.000719 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10e0194-58ec-4ad7-a8eb-bc747cdb93eb admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:37.011199 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1202/2423] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:18:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:18:37.015522 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca7db37-97ab-4299-9117-0c46c4df1565 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:37.025884 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca7db37-97ab-4299-9117-0c46c4df1565 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:37.030505 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e7bb9c-5cde-4d67-98fe-bc1f4b385c48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:37.031210 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e7bb9c-5cde-4d67-98fe-bc1f4b385c48 admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:37.031491 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e7bb9c-5cde-4d67-98fe-bc1f4b385c48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:37.031763 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e7bb9c-5cde-4d67-98fe-bc1f4b385c48 admin admin] PATH_INFO: `/v3/projects/78ba49ec638f4f0982fb5b7b13fe5457/users/d3b749b04e064904879e1078fe49aeb6/roles/eecd2f160dd844f5906f3d345780c501` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:37.046859 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e7bb9c-5cde-4d67-98fe-bc1f4b385c48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78ba49ec638f4f0982fb5b7b13fe5457, user_id=d3b749b04e064904879e1078fe49aeb6, role_id=eecd2f160dd844f5906f3d345780c501)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:37.049545 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e7bb9c-5cde-4d67-98fe-bc1f4b385c48 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:37.063394 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1222/2424] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:18:37 2022] PUT /identity/v3/projects/78ba49ec638f4f0982fb5b7b13fe5457/users/d3b749b04e064904879e1078fe49aeb6/roles/eecd2f160dd844f5906f3d345780c501 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:18:37.079931 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67138cba-2175-4e09-b7a5-cb7d6c853111 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:37.080214 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67138cba-2175-4e09-b7a5-cb7d6c853111 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:37.080486 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67138cba-2175-4e09-b7a5-cb7d6c853111 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:37.109350 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-67138cba-2175-4e09-b7a5-cb7d6c853111 None None] MFA Rules not processed for user `d3b749b04e064904879e1078fe49aeb6`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:18:37.110592 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-67138cba-2175-4e09-b7a5-cb7d6c853111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:18:37.173701 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1203/2425] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:18:37 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:18:37.178358 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e7bb9c-5cde-4d67-98fe-bc1f4b385c48 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:37.238846 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e7bb9c-5cde-4d67-98fe-bc1f4b385c48 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:37.243912 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b60e2e-e034-4b24-93ee-676dfaeceadc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:37.244550 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b60e2e-e034-4b24-93ee-676dfaeceadc admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:37.244820 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b60e2e-e034-4b24-93ee-676dfaeceadc admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:37.245090 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b60e2e-e034-4b24-93ee-676dfaeceadc admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:37.245719 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b60e2e-e034-4b24-93ee-676dfaeceadc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:37.247791 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b60e2e-e034-4b24-93ee-676dfaeceadc admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:37.263314 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:18:37.281299 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67138cba-2175-4e09-b7a5-cb7d6c853111 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:37.284124 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1223/2426] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 9 12:18:37 2022] POST /identity/v3/projects => generated 363 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:18:37.292336 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67138cba-2175-4e09-b7a5-cb7d6c853111 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:37.351352 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cda1631-8f4b-47e5-b3ef-998c8971e85b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:37.351961 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cda1631-8f4b-47e5-b3ef-998c8971e85b admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:37.352238 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cda1631-8f4b-47e5-b3ef-998c8971e85b admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:37.352505 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cda1631-8f4b-47e5-b3ef-998c8971e85b admin admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:37.353319 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cda1631-8f4b-47e5-b3ef-998c8971e85b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:37.355420 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cda1631-8f4b-47e5-b3ef-998c8971e85b admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:37.380034 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1204/2427] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:18:37 2022] POST /identity/v3/users => generated 288 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:18:37.381659 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b60e2e-e034-4b24-93ee-676dfaeceadc admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:37.393057 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b60e2e-e034-4b24-93ee-676dfaeceadc admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:37.399554 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f975031-7fd7-476a-ba81-d4120aa44ff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:37.400838 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f975031-7fd7-476a-ba81-d4120aa44ff6 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:37.401579 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f975031-7fd7-476a-ba81-d4120aa44ff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:37.402010 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f975031-7fd7-476a-ba81-d4120aa44ff6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:37.402721 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f975031-7fd7-476a-ba81-d4120aa44ff6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:37.404409 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f975031-7fd7-476a-ba81-d4120aa44ff6 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:37.415257 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1224/2428] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:18:37 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:18:37.419564 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cda1631-8f4b-47e5-b3ef-998c8971e85b admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:37.429670 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cda1631-8f4b-47e5-b3ef-998c8971e85b admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:37.434407 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1deacf7c-3bbe-4848-a20d-b722aed68844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:37.435086 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deacf7c-3bbe-4848-a20d-b722aed68844 admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:37.435366 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deacf7c-3bbe-4848-a20d-b722aed68844 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:37.435638 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deacf7c-3bbe-4848-a20d-b722aed68844 admin admin] PATH_INFO: `/v3/projects/9e6e39c7963e4c39999b4cd7cabe8d88/users/73836ba38a2443ec9db30c7fb382a50e/roles/3f2ca33902a04b6dbc2ef7d06d7289d3` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:37.450496 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1deacf7c-3bbe-4848-a20d-b722aed68844 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e6e39c7963e4c39999b4cd7cabe8d88, user_id=73836ba38a2443ec9db30c7fb382a50e, role_id=3f2ca33902a04b6dbc2ef7d06d7289d3)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:37.453195 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1deacf7c-3bbe-4848-a20d-b722aed68844 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:37.462346 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:18:37.467715 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1205/2429] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:18:37 2022] PUT /identity/v3/projects/9e6e39c7963e4c39999b4cd7cabe8d88/users/73836ba38a2443ec9db30c7fb382a50e/roles/3f2ca33902a04b6dbc2ef7d06d7289d3 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:18:37.471808 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f975031-7fd7-476a-ba81-d4120aa44ff6 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:37.481592 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f975031-7fd7-476a-ba81-d4120aa44ff6 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:37.537340 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43ed78c-906b-400d-b311-b231c81aca3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:37.537942 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43ed78c-906b-400d-b311-b231c81aca3f admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:37.538219 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43ed78c-906b-400d-b311-b231c81aca3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:37.538490 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43ed78c-906b-400d-b311-b231c81aca3f admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:37.539149 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43ed78c-906b-400d-b311-b231c81aca3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:37.540810 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43ed78c-906b-400d-b311-b231c81aca3f admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:37.550872 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1225/2430] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:18:37 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:18:37.555307 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1deacf7c-3bbe-4848-a20d-b722aed68844 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:37.565463 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1deacf7c-3bbe-4848-a20d-b722aed68844 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:37.570082 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0911b9ad-4c8e-47a2-aa53-5bc6e8368815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:37.570730 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0911b9ad-4c8e-47a2-aa53-5bc6e8368815 admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:37.571056 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0911b9ad-4c8e-47a2-aa53-5bc6e8368815 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:37.571315 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0911b9ad-4c8e-47a2-aa53-5bc6e8368815 admin admin] PATH_INFO: `/v3/projects/9e6e39c7963e4c39999b4cd7cabe8d88/users/73836ba38a2443ec9db30c7fb382a50e/roles/3f2ca33902a04b6dbc2ef7d06d7289d3` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:37.573730 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0911b9ad-4c8e-47a2-aa53-5bc6e8368815 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e6e39c7963e4c39999b4cd7cabe8d88, user_id=73836ba38a2443ec9db30c7fb382a50e, role_id=3f2ca33902a04b6dbc2ef7d06d7289d3)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:37.576440 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0911b9ad-4c8e-47a2-aa53-5bc6e8368815 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:37.586472 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1206/2431] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:18:37 2022] PUT /identity/v3/projects/9e6e39c7963e4c39999b4cd7cabe8d88/users/73836ba38a2443ec9db30c7fb382a50e/roles/3f2ca33902a04b6dbc2ef7d06d7289d3 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:18:37.597342 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939013f5-f567-4140-bb26-afb10458254f None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:37.597695 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939013f5-f567-4140-bb26-afb10458254f None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:37.597960 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939013f5-f567-4140-bb26-afb10458254f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:37.630040 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-939013f5-f567-4140-bb26-afb10458254f None None] MFA Rules not processed for user `73836ba38a2443ec9db30c7fb382a50e`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:18:37.631282 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-939013f5-f567-4140-bb26-afb10458254f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:18:37.706047 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1226/2432] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:18:37 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:18:37.717426 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0911b9ad-4c8e-47a2-aa53-5bc6e8368815 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:37.719208 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-0911b9ad-4c8e-47a2-aa53-5bc6e8368815 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:18:37.777699 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0911b9ad-4c8e-47a2-aa53-5bc6e8368815 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:37.781441 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119549ab-779b-41c0-bc93-992bebb482ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535c258100df40b58a5ee44b2c21d621', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:37.782097 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119549ab-779b-41c0-bc93-992bebb482ab service cinder] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:37.782321 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119549ab-779b-41c0-bc93-992bebb482ab service cinder] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:37.782586 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119549ab-779b-41c0-bc93-992bebb482ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:37.791968 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119549ab-779b-41c0-bc93-992bebb482ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:37.793393 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119549ab-779b-41c0-bc93-992bebb482ab service cinder] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:37.802976 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1207/2433] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:18:37 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:18:37.880631 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939013f5-f567-4140-bb26-afb10458254f None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:37.890990 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939013f5-f567-4140-bb26-afb10458254f None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:37.894592 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8c6e2a-9dea-4941-9569-6fb09404a295 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535c258100df40b58a5ee44b2c21d621', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:37.895219 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8c6e2a-9dea-4941-9569-6fb09404a295 service cinder] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:37.896064 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8c6e2a-9dea-4941-9569-6fb09404a295 service cinder] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:37.896064 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8c6e2a-9dea-4941-9569-6fb09404a295 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:37.953550 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8c6e2a-9dea-4941-9569-6fb09404a295 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:37.955042 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8c6e2a-9dea-4941-9569-6fb09404a295 service cinder] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:37.964392 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1227/2434] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:18:37 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:18:39.754473 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7acee34-26f6-4163-8359-759e55cd9e13 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:39.755144 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7acee34-26f6-4163-8359-759e55cd9e13 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:39.755384 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7acee34-26f6-4163-8359-759e55cd9e13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:39.777643 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-c7acee34-26f6-4163-8359-759e55cd9e13 None None] MFA Rules not processed for user `e368b4e668c54e25bff80a8e8a8c3588`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:18:39.779050 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-c7acee34-26f6-4163-8359-759e55cd9e13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:18:39.841383 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1208/2435] 10.222.0.49 () {58 vars in 967 bytes} [Fri Sep 9 12:18:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:18:39.889518 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8c6e2a-9dea-4941-9569-6fb09404a295 service cinder] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:39.890938 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-dd8c6e2a-9dea-4941-9569-6fb09404a295 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:18:39.897439 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:18:39.959416 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8c6e2a-9dea-4941-9569-6fb09404a295 service cinder] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:39.964599 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec9b12c-365e-4b40-8eb3-a8c2edab3783 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e652ba827849b5a3fc1213d04a1b76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:39.965197 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec9b12c-365e-4b40-8eb3-a8c2edab3783 service swift] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:39.965472 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec9b12c-365e-4b40-8eb3-a8c2edab3783 service swift] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:39.965740 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec9b12c-365e-4b40-8eb3-a8c2edab3783 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:39.973837 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec9b12c-365e-4b40-8eb3-a8c2edab3783 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:39.975549 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec9b12c-365e-4b40-8eb3-a8c2edab3783 service swift] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:39.984158 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1228/2436] 10.222.0.49 () {60 vars in 1384 bytes} [Fri Sep 9 12:18:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 12:18:40.730210 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7acee34-26f6-4163-8359-759e55cd9e13 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:40.795402 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7acee34-26f6-4163-8359-759e55cd9e13 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:40.799968 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6941098-3f55-4e79-b6ba-9beddd91641e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:40.800594 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6941098-3f55-4e79-b6ba-9beddd91641e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:40.800866 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6941098-3f55-4e79-b6ba-9beddd91641e admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:40.801133 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6941098-3f55-4e79-b6ba-9beddd91641e admin admin] PATH_INFO: `/v3/users/d3b749b04e064904879e1078fe49aeb6` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:40.802276 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6941098-3f55-4e79-b6ba-9beddd91641e admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3b749b04e064904879e1078fe49aeb6)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:40.804487 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6941098-3f55-4e79-b6ba-9beddd91641e admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:40.856260 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:18:40.876612 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-c6941098-3f55-4e79-b6ba-9beddd91641e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3b749b04e064904879e1078fe49aeb6'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:40.893504 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-c6941098-3f55-4e79-b6ba-9beddd91641e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3b749b04e064904879e1078fe49aeb6'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:40.906331 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-c6941098-3f55-4e79-b6ba-9beddd91641e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3b749b04e064904879e1078fe49aeb6'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:40.915785 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-c6941098-3f55-4e79-b6ba-9beddd91641e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3b749b04e064904879e1078fe49aeb6'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:40.916576 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-c6941098-3f55-4e79-b6ba-9beddd91641e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3b749b04e064904879e1078fe49aeb6'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:40.929292 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1209/2437] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 9 12:18:40 2022] DELETE /identity/v3/users/d3b749b04e064904879e1078fe49aeb6 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:18:40.932460 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bab30f-e841-41ea-87fc-ff7f37b62987 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:40.932951 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bab30f-e841-41ea-87fc-ff7f37b62987 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:40.933207 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bab30f-e841-41ea-87fc-ff7f37b62987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:40.956129 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-d7bab30f-e841-41ea-87fc-ff7f37b62987 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:18:40.957517 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-d7bab30f-e841-41ea-87fc-ff7f37b62987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:18:41.043966 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1229/2438] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:18:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:18:41.056780 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6941098-3f55-4e79-b6ba-9beddd91641e admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:41.059148 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-c6941098-3f55-4e79-b6ba-9beddd91641e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:18:41.134386 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6941098-3f55-4e79-b6ba-9beddd91641e admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:41.138160 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d9b3d1-bdd7-44cf-a55a-d6d25aea87c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:41.138767 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d9b3d1-bdd7-44cf-a55a-d6d25aea87c8 service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:41.139082 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d9b3d1-bdd7-44cf-a55a-d6d25aea87c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:41.139362 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d9b3d1-bdd7-44cf-a55a-d6d25aea87c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:41.149424 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d9b3d1-bdd7-44cf-a55a-d6d25aea87c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:41.151154 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d9b3d1-bdd7-44cf-a55a-d6d25aea87c8 service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:41.160882 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1210/2439] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:18:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:18:41.449137 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bab30f-e841-41ea-87fc-ff7f37b62987 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:41.450825 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-d7bab30f-e841-41ea-87fc-ff7f37b62987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:18:41.468159 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bab30f-e841-41ea-87fc-ff7f37b62987 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:41.474433 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2626b8-6809-46f5-988f-84f5c75f6b87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:41.475361 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2626b8-6809-46f5-988f-84f5c75f6b87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:41.475840 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2626b8-6809-46f5-988f-84f5c75f6b87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:41.476305 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2626b8-6809-46f5-988f-84f5c75f6b87 admin admin] PATH_INFO: `/v3/projects/78ba49ec638f4f0982fb5b7b13fe5457` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:41.480747 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2626b8-6809-46f5-988f-84f5c75f6b87 admin admin] RBAC: Authorizing `identity:delete_project(project_id=78ba49ec638f4f0982fb5b7b13fe5457)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:41.482668 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2626b8-6809-46f5-988f-84f5c75f6b87 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:41.532156 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-ef2626b8-6809-46f5-988f-84f5c75f6b87 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78ba49ec638f4f0982fb5b7b13fe5457', 'request_id': 'req-ef2626b8-6809-46f5-988f-84f5c75f6b87'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:41.538251 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-ef2626b8-6809-46f5-988f-84f5c75f6b87 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78ba49ec638f4f0982fb5b7b13fe5457', 'request_id': 'req-ef2626b8-6809-46f5-988f-84f5c75f6b87'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:41.545447 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-ef2626b8-6809-46f5-988f-84f5c75f6b87 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78ba49ec638f4f0982fb5b7b13fe5457', 'request_id': 'req-ef2626b8-6809-46f5-988f-84f5c75f6b87'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:41.559295 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-ef2626b8-6809-46f5-988f-84f5c75f6b87 admin admin] The token cache is being invalidate because project 78ba49ec638f4f0982fb5b7b13fe5457 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81591) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:18:41.559693 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-ef2626b8-6809-46f5-988f-84f5c75f6b87 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:41.560340 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-ef2626b8-6809-46f5-988f-84f5c75f6b87 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:41.561851 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1230/2440] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 9 12:18:41 2022] DELETE /identity/v3/projects/78ba49ec638f4f0982fb5b7b13fe5457 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:18:41.565957 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d9b3d1-bdd7-44cf-a55a-d6d25aea87c8 service neutron] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:41.576285 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d9b3d1-bdd7-44cf-a55a-d6d25aea87c8 service neutron] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:41.640107 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5fe8d5-754f-4456-812b-b4d68e9cfa3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:41.640750 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5fe8d5-754f-4456-812b-b4d68e9cfa3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:41.641032 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5fe8d5-754f-4456-812b-b4d68e9cfa3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:41.641308 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5fe8d5-754f-4456-812b-b4d68e9cfa3f admin admin] PATH_INFO: `/v3/users/73836ba38a2443ec9db30c7fb382a50e` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:41.642461 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5fe8d5-754f-4456-812b-b4d68e9cfa3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=73836ba38a2443ec9db30c7fb382a50e)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:41.644446 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5fe8d5-754f-4456-812b-b4d68e9cfa3f admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:41.694625 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-ea5fe8d5-754f-4456-812b-b4d68e9cfa3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73836ba38a2443ec9db30c7fb382a50e'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:41.709309 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:18:41.713505 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-ea5fe8d5-754f-4456-812b-b4d68e9cfa3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73836ba38a2443ec9db30c7fb382a50e'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:41.723921 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-ea5fe8d5-754f-4456-812b-b4d68e9cfa3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73836ba38a2443ec9db30c7fb382a50e'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:41.732717 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-ea5fe8d5-754f-4456-812b-b4d68e9cfa3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73836ba38a2443ec9db30c7fb382a50e'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:41.733405 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-ea5fe8d5-754f-4456-812b-b4d68e9cfa3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73836ba38a2443ec9db30c7fb382a50e'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:41.741298 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1211/2441] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 9 12:18:41 2022] DELETE /identity/v3/users/73836ba38a2443ec9db30c7fb382a50e => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:18:42.221205 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2626b8-6809-46f5-988f-84f5c75f6b87 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:42.222738 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-ef2626b8-6809-46f5-988f-84f5c75f6b87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:18:42.237393 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2626b8-6809-46f5-988f-84f5c75f6b87 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:42.257073 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:18:42.305810 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392f1a9a-d7ca-41d8-b7b1-354ff84662ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:42.307313 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392f1a9a-d7ca-41d8-b7b1-354ff84662ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:42.307601 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392f1a9a-d7ca-41d8-b7b1-354ff84662ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:42.307869 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392f1a9a-d7ca-41d8-b7b1-354ff84662ee admin admin] PATH_INFO: `/v3/projects/9e6e39c7963e4c39999b4cd7cabe8d88` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:42.309142 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392f1a9a-d7ca-41d8-b7b1-354ff84662ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e6e39c7963e4c39999b4cd7cabe8d88)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:42.311304 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392f1a9a-d7ca-41d8-b7b1-354ff84662ee admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:42.361510 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-392f1a9a-d7ca-41d8-b7b1-354ff84662ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e6e39c7963e4c39999b4cd7cabe8d88', 'request_id': 'req-392f1a9a-d7ca-41d8-b7b1-354ff84662ee'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:42.367375 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-392f1a9a-d7ca-41d8-b7b1-354ff84662ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e6e39c7963e4c39999b4cd7cabe8d88', 'request_id': 'req-392f1a9a-d7ca-41d8-b7b1-354ff84662ee'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:42.374313 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-392f1a9a-d7ca-41d8-b7b1-354ff84662ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e6e39c7963e4c39999b4cd7cabe8d88', 'request_id': 'req-392f1a9a-d7ca-41d8-b7b1-354ff84662ee'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:42.390204 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-392f1a9a-d7ca-41d8-b7b1-354ff84662ee admin admin] The token cache is being invalidate because project 9e6e39c7963e4c39999b4cd7cabe8d88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81591) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:18:42.391310 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-392f1a9a-d7ca-41d8-b7b1-354ff84662ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:42.392164 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-392f1a9a-d7ca-41d8-b7b1-354ff84662ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:42.394344 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1231/2442] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 9 12:18:42 2022] DELETE /identity/v3/projects/9e6e39c7963e4c39999b4cd7cabe8d88 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:18:42.399271 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3c7a6a-e6a0-43b2-a01b-d50bb129f25a None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:42.399747 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3c7a6a-e6a0-43b2-a01b-d50bb129f25a None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:42.400086 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3c7a6a-e6a0-43b2-a01b-d50bb129f25a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:42.432421 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-7f3c7a6a-e6a0-43b2-a01b-d50bb129f25a None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:18:42.433675 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-7f3c7a6a-e6a0-43b2-a01b-d50bb129f25a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:18:42.508553 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1212/2443] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:18:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:18:42.517996 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec3f782-0743-4ce5-a848-863c028cf4b2 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:42.518283 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec3f782-0743-4ce5-a848-863c028cf4b2 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:42.518557 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec3f782-0743-4ce5-a848-863c028cf4b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:42.540237 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-6ec3f782-0743-4ce5-a848-863c028cf4b2 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:18:42.542249 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-6ec3f782-0743-4ce5-a848-863c028cf4b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:18:42.550855 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1232/2444] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:18:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:18:42.555111 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3c7a6a-e6a0-43b2-a01b-d50bb129f25a None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:42.571101 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3c7a6a-e6a0-43b2-a01b-d50bb129f25a None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:42.583044 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9b84ef-01ad-4df2-bbce-2a927cc7a39c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:42.583742 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9b84ef-01ad-4df2-bbce-2a927cc7a39c admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:42.583956 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9b84ef-01ad-4df2-bbce-2a927cc7a39c admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:42.584205 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9b84ef-01ad-4df2-bbce-2a927cc7a39c admin admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:42.584858 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9b84ef-01ad-4df2-bbce-2a927cc7a39c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:18:42.585162 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9b84ef-01ad-4df2-bbce-2a927cc7a39c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:42.586792 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9b84ef-01ad-4df2-bbce-2a927cc7a39c admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:42.600911 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1213/2445] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 9 12:18:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:18:42.605559 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec3f782-0743-4ce5-a848-863c028cf4b2 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:42.615652 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec3f782-0743-4ce5-a848-863c028cf4b2 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:42.623727 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa4c6aa-1143-4baf-988f-6440cf3e05b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:42.624654 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa4c6aa-1143-4baf-988f-6440cf3e05b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:42.625158 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa4c6aa-1143-4baf-988f-6440cf3e05b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:42.625657 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa4c6aa-1143-4baf-988f-6440cf3e05b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:42.626528 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa4c6aa-1143-4baf-988f-6440cf3e05b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:42.628765 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa4c6aa-1143-4baf-988f-6440cf3e05b9 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:42.662303 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1233/2446] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 9 12:18:42 2022] POST /identity/v3/projects => generated 367 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:18:42.667154 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9b84ef-01ad-4df2-bbce-2a927cc7a39c admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:42.677439 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9b84ef-01ad-4df2-bbce-2a927cc7a39c admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:42.764902 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67882dc-bd25-4043-8fe0-f9e24276883d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:42.765534 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67882dc-bd25-4043-8fe0-f9e24276883d admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:42.765810 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67882dc-bd25-4043-8fe0-f9e24276883d admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:42.766710 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67882dc-bd25-4043-8fe0-f9e24276883d admin admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:42.767397 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67882dc-bd25-4043-8fe0-f9e24276883d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:42.769682 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67882dc-bd25-4043-8fe0-f9e24276883d admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:42.806755 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1214/2447] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:18:42 2022] POST /identity/v3/users => generated 284 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:18:42.811217 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa4c6aa-1143-4baf-988f-6440cf3e05b9 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:42.826289 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa4c6aa-1143-4baf-988f-6440cf3e05b9 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:42.833868 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013eb393-5a74-495a-9eff-7e75d81f90ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:42.834581 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013eb393-5a74-495a-9eff-7e75d81f90ac admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:42.835459 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013eb393-5a74-495a-9eff-7e75d81f90ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:42.835810 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013eb393-5a74-495a-9eff-7e75d81f90ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:42.836590 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013eb393-5a74-495a-9eff-7e75d81f90ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:42.838961 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013eb393-5a74-495a-9eff-7e75d81f90ac admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:42.861353 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67882dc-bd25-4043-8fe0-f9e24276883d admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:42.866924 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1234/2448] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:18:42 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:18:42.874840 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67882dc-bd25-4043-8fe0-f9e24276883d admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:42.883126 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1707b211-c5c8-487c-afb8-f5d8b5c02bef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:42.883911 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1707b211-c5c8-487c-afb8-f5d8b5c02bef admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:42.884297 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1707b211-c5c8-487c-afb8-f5d8b5c02bef admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:42.884690 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1707b211-c5c8-487c-afb8-f5d8b5c02bef admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:42.885516 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1707b211-c5c8-487c-afb8-f5d8b5c02bef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:42.887275 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1707b211-c5c8-487c-afb8-f5d8b5c02bef admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:42.903283 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1215/2449] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:18:42 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:18:42.904920 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013eb393-5a74-495a-9eff-7e75d81f90ac admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:42.919161 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013eb393-5a74-495a-9eff-7e75d81f90ac admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:42.926227 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ef394e-28fb-452c-ad17-cc646b5b538e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:42.926925 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ef394e-28fb-452c-ad17-cc646b5b538e admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:42.927218 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ef394e-28fb-452c-ad17-cc646b5b538e admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:42.927504 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ef394e-28fb-452c-ad17-cc646b5b538e admin admin] PATH_INFO: `/v3/projects/7c59ff6b62944c88938bc02bf308f1b7/users/e3b73b56a22f4e21b7d3ee207117ab6f/roles/eecd2f160dd844f5906f3d345780c501` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:42.943333 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ef394e-28fb-452c-ad17-cc646b5b538e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c59ff6b62944c88938bc02bf308f1b7, user_id=e3b73b56a22f4e21b7d3ee207117ab6f, role_id=eecd2f160dd844f5906f3d345780c501)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:42.946892 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ef394e-28fb-452c-ad17-cc646b5b538e admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:42.964309 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1235/2450] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:18:42 2022] PUT /identity/v3/projects/7c59ff6b62944c88938bc02bf308f1b7/users/e3b73b56a22f4e21b7d3ee207117ab6f/roles/eecd2f160dd844f5906f3d345780c501 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:18:42.974562 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1738ac45-e868-4818-88ad-40224c283281 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:42.974856 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1738ac45-e868-4818-88ad-40224c283281 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:42.975157 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1738ac45-e868-4818-88ad-40224c283281 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:43.006963 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-1738ac45-e868-4818-88ad-40224c283281 None None] MFA Rules not processed for user `e3b73b56a22f4e21b7d3ee207117ab6f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:18:43.008809 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-1738ac45-e868-4818-88ad-40224c283281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:18:43.050735 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:18:43.112932 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1216/2451] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:18:42 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:18:43.117449 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ef394e-28fb-452c-ad17-cc646b5b538e admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:43.176995 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:18:43.197852 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ef394e-28fb-452c-ad17-cc646b5b538e admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:43.204866 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88c9f78-bb49-4076-8b6b-a0f2f6d9e509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:43.204866 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88c9f78-bb49-4076-8b6b-a0f2f6d9e509 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:43.205167 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88c9f78-bb49-4076-8b6b-a0f2f6d9e509 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:43.205549 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88c9f78-bb49-4076-8b6b-a0f2f6d9e509 admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:43.206271 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88c9f78-bb49-4076-8b6b-a0f2f6d9e509 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:43.208332 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88c9f78-bb49-4076-8b6b-a0f2f6d9e509 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:43.245775 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1236/2452] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 9 12:18:43 2022] POST /identity/v3/projects => generated 367 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:18:43.246930 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1738ac45-e868-4818-88ad-40224c283281 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:43.261128 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1738ac45-e868-4818-88ad-40224c283281 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:43.325903 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19552436-7430-4c1c-a1a4-a9039434d6cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:43.326610 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19552436-7430-4c1c-a1a4-a9039434d6cf admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:43.326978 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19552436-7430-4c1c-a1a4-a9039434d6cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:43.327323 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19552436-7430-4c1c-a1a4-a9039434d6cf admin admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:43.328065 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19552436-7430-4c1c-a1a4-a9039434d6cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:43.330000 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19552436-7430-4c1c-a1a4-a9039434d6cf admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:43.354552 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1217/2453] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:18:43 2022] POST /identity/v3/users => generated 290 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:18:43.358772 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88c9f78-bb49-4076-8b6b-a0f2f6d9e509 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:43.377587 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88c9f78-bb49-4076-8b6b-a0f2f6d9e509 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:43.385432 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ac35bb-48ec-4f6d-afe7-63ec40ce8f09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:43.386284 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ac35bb-48ec-4f6d-afe7-63ec40ce8f09 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:43.386592 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ac35bb-48ec-4f6d-afe7-63ec40ce8f09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:43.386863 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ac35bb-48ec-4f6d-afe7-63ec40ce8f09 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:43.387539 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ac35bb-48ec-4f6d-afe7-63ec40ce8f09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:43.389369 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ac35bb-48ec-4f6d-afe7-63ec40ce8f09 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:43.411231 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1237/2454] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:18:43 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:18:43.415520 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19552436-7430-4c1c-a1a4-a9039434d6cf admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:43.429919 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19552436-7430-4c1c-a1a4-a9039434d6cf admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:43.436720 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e563fb74-3a7c-4e20-bd5c-580cb7ef8da4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:43.437451 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e563fb74-3a7c-4e20-bd5c-580cb7ef8da4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:43.437664 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e563fb74-3a7c-4e20-bd5c-580cb7ef8da4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:43.437941 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e563fb74-3a7c-4e20-bd5c-580cb7ef8da4 admin admin] PATH_INFO: `/v3/projects/1a9c4c78c0114854a50795dac49fbc49/users/09826346081749b4956661e7172ae5c4/roles/3f2ca33902a04b6dbc2ef7d06d7289d3` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:43.458548 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e563fb74-3a7c-4e20-bd5c-580cb7ef8da4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a9c4c78c0114854a50795dac49fbc49, user_id=09826346081749b4956661e7172ae5c4, role_id=3f2ca33902a04b6dbc2ef7d06d7289d3)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:43.461332 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e563fb74-3a7c-4e20-bd5c-580cb7ef8da4 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:43.476768 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1218/2455] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:18:43 2022] PUT /identity/v3/projects/1a9c4c78c0114854a50795dac49fbc49/users/09826346081749b4956661e7172ae5c4/roles/3f2ca33902a04b6dbc2ef7d06d7289d3 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:18:43.481010 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ac35bb-48ec-4f6d-afe7-63ec40ce8f09 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:43.491366 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ac35bb-48ec-4f6d-afe7-63ec40ce8f09 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:43.558403 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf3c242-d774-460b-90bf-048b249b58ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:43.559174 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf3c242-d774-460b-90bf-048b249b58ca admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:43.559522 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf3c242-d774-460b-90bf-048b249b58ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:43.559977 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf3c242-d774-460b-90bf-048b249b58ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:43.561102 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf3c242-d774-460b-90bf-048b249b58ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:43.562352 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf3c242-d774-460b-90bf-048b249b58ca admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:43.574994 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1238/2456] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:18:43 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:18:43.579588 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e563fb74-3a7c-4e20-bd5c-580cb7ef8da4 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:43.592119 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e563fb74-3a7c-4e20-bd5c-580cb7ef8da4 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:43.597045 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e7ea96-41c2-4ac8-b727-d3348f59f695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:43.597731 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e7ea96-41c2-4ac8-b727-d3348f59f695 admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:43.598008 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e7ea96-41c2-4ac8-b727-d3348f59f695 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:43.598288 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e7ea96-41c2-4ac8-b727-d3348f59f695 admin admin] PATH_INFO: `/v3/projects/1a9c4c78c0114854a50795dac49fbc49/users/09826346081749b4956661e7172ae5c4/roles/3f2ca33902a04b6dbc2ef7d06d7289d3` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:43.600223 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e7ea96-41c2-4ac8-b727-d3348f59f695 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a9c4c78c0114854a50795dac49fbc49, user_id=09826346081749b4956661e7172ae5c4, role_id=3f2ca33902a04b6dbc2ef7d06d7289d3)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:43.602868 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e7ea96-41c2-4ac8-b727-d3348f59f695 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:43.617251 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1219/2457] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:18:43 2022] PUT /identity/v3/projects/1a9c4c78c0114854a50795dac49fbc49/users/09826346081749b4956661e7172ae5c4/roles/3f2ca33902a04b6dbc2ef7d06d7289d3 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:18:43.629296 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c9cd2b-7f13-4cfb-8f63-f895d5b7e1d0 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:43.629586 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c9cd2b-7f13-4cfb-8f63-f895d5b7e1d0 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:43.629855 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c9cd2b-7f13-4cfb-8f63-f895d5b7e1d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:43.661234 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-68c9cd2b-7f13-4cfb-8f63-f895d5b7e1d0 None None] MFA Rules not processed for user `09826346081749b4956661e7172ae5c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:18:43.662701 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-68c9cd2b-7f13-4cfb-8f63-f895d5b7e1d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:18:43.746553 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1239/2458] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:18:43 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:18:43.761265 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e7ea96-41c2-4ac8-b727-d3348f59f695 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:43.763059 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-32e7ea96-41c2-4ac8-b727-d3348f59f695 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:18:43.839433 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e7ea96-41c2-4ac8-b727-d3348f59f695 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:43.846773 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f4c81a-a54f-43b5-b33a-6dddc227caa4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535c258100df40b58a5ee44b2c21d621', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:43.847535 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f4c81a-a54f-43b5-b33a-6dddc227caa4 service cinder] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:43.847820 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f4c81a-a54f-43b5-b33a-6dddc227caa4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:43.848095 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f4c81a-a54f-43b5-b33a-6dddc227caa4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:43.859378 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f4c81a-a54f-43b5-b33a-6dddc227caa4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:43.860840 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f4c81a-a54f-43b5-b33a-6dddc227caa4 service cinder] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:43.875334 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1220/2459] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:18:43 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:18:43.932122 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c9cd2b-7f13-4cfb-8f63-f895d5b7e1d0 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:43.933645 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-68c9cd2b-7f13-4cfb-8f63-f895d5b7e1d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:18:43.945272 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c9cd2b-7f13-4cfb-8f63-f895d5b7e1d0 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:43.949647 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f93942a-d16d-48a3-a631-9f320501f60e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535c258100df40b58a5ee44b2c21d621', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:43.950267 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f93942a-d16d-48a3-a631-9f320501f60e service cinder] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:43.950577 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f93942a-d16d-48a3-a631-9f320501f60e service cinder] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:43.950926 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f93942a-d16d-48a3-a631-9f320501f60e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:43.990419 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:18:44.017533 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f93942a-d16d-48a3-a631-9f320501f60e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:44.018985 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f93942a-d16d-48a3-a631-9f320501f60e service cinder] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:44.028295 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1240/2460] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:18:43 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:18:47.645817 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f4c81a-a54f-43b5-b33a-6dddc227caa4 service cinder] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:47.679230 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:18:47.725303 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f4c81a-a54f-43b5-b33a-6dddc227caa4 service cinder] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:47.730206 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f31ed76-9df1-4b80-b171-f0b05019bd3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:47.731454 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f31ed76-9df1-4b80-b171-f0b05019bd3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:47.731454 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f31ed76-9df1-4b80-b171-f0b05019bd3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:47.731454 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f31ed76-9df1-4b80-b171-f0b05019bd3c admin admin] PATH_INFO: `/v3/users/e3b73b56a22f4e21b7d3ee207117ab6f` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:47.732531 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f31ed76-9df1-4b80-b171-f0b05019bd3c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3b73b56a22f4e21b7d3ee207117ab6f)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:47.734641 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f31ed76-9df1-4b80-b171-f0b05019bd3c admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:47.820316 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f93942a-d16d-48a3-a631-9f320501f60e service cinder] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:47.822432 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-6f31ed76-9df1-4b80-b171-f0b05019bd3c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3b73b56a22f4e21b7d3ee207117ab6f'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:47.822562 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-4f93942a-d16d-48a3-a631-9f320501f60e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:18:47.841119 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-6f31ed76-9df1-4b80-b171-f0b05019bd3c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3b73b56a22f4e21b7d3ee207117ab6f'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:47.844423 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f93942a-d16d-48a3-a631-9f320501f60e service cinder] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:47.853786 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b383904-01ac-4677-9b6a-e964d932b83c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:47.854535 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b383904-01ac-4677-9b6a-e964d932b83c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:47.854876 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b383904-01ac-4677-9b6a-e964d932b83c admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:47.855068 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-6f31ed76-9df1-4b80-b171-f0b05019bd3c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3b73b56a22f4e21b7d3ee207117ab6f'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:47.855317 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b383904-01ac-4677-9b6a-e964d932b83c admin admin] PATH_INFO: `/v3/users/eb42a8cde3044a4b9df5cb81d82d9910` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:47.856533 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b383904-01ac-4677-9b6a-e964d932b83c admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb42a8cde3044a4b9df5cb81d82d9910)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:47.858608 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b383904-01ac-4677-9b6a-e964d932b83c admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:47.863325 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-6f31ed76-9df1-4b80-b171-f0b05019bd3c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3b73b56a22f4e21b7d3ee207117ab6f'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:47.864138 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-6f31ed76-9df1-4b80-b171-f0b05019bd3c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3b73b56a22f4e21b7d3ee207117ab6f'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:47.870702 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1221/2461] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 9 12:18:47 2022] DELETE /identity/v3/users/e3b73b56a22f4e21b7d3ee207117ab6f => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:18:47.874793 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdad860-6d6e-4c27-aa0a-4d3a6bac1830 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:47.875126 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdad860-6d6e-4c27-aa0a-4d3a6bac1830 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:47.875402 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdad860-6d6e-4c27-aa0a-4d3a6bac1830 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:47.896620 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-ffdad860-6d6e-4c27-aa0a-4d3a6bac1830 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:18:47.897984 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-ffdad860-6d6e-4c27-aa0a-4d3a6bac1830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:18:47.911970 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-5b383904-01ac-4677-9b6a-e964d932b83c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb42a8cde3044a4b9df5cb81d82d9910'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:47.919161 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-5b383904-01ac-4677-9b6a-e964d932b83c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb42a8cde3044a4b9df5cb81d82d9910'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:47.931984 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-5b383904-01ac-4677-9b6a-e964d932b83c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb42a8cde3044a4b9df5cb81d82d9910'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:47.939860 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-5b383904-01ac-4677-9b6a-e964d932b83c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb42a8cde3044a4b9df5cb81d82d9910'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:47.940501 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-5b383904-01ac-4677-9b6a-e964d932b83c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb42a8cde3044a4b9df5cb81d82d9910'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:47.947983 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1241/2462] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 9 12:18:47 2022] DELETE /identity/v3/users/eb42a8cde3044a4b9df5cb81d82d9910 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:18:47.960065 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1222/2463] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:18:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:18:47.970648 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b383904-01ac-4677-9b6a-e964d932b83c admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:47.972105 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-5b383904-01ac-4677-9b6a-e964d932b83c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:18:48.031909 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b383904-01ac-4677-9b6a-e964d932b83c admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:48.035356 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8f3afc-f403-40d0-a4f3-02a879c8aa14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:48.035972 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8f3afc-f403-40d0-a4f3-02a879c8aa14 service neutron] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:48.036223 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8f3afc-f403-40d0-a4f3-02a879c8aa14 service neutron] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:48.036491 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8f3afc-f403-40d0-a4f3-02a879c8aa14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:48.104634 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8f3afc-f403-40d0-a4f3-02a879c8aa14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:48.106070 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8f3afc-f403-40d0-a4f3-02a879c8aa14 service neutron] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:48.114070 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:18:48.121477 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1242/2464] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:18:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:18:48.136007 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdad860-6d6e-4c27-aa0a-4d3a6bac1830 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:48.137406 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-ffdad860-6d6e-4c27-aa0a-4d3a6bac1830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:18:48.156612 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdad860-6d6e-4c27-aa0a-4d3a6bac1830 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:48.163821 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6526c6d2-d15c-4747-8cbd-a3cca208a260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:48.164467 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6526c6d2-d15c-4747-8cbd-a3cca208a260 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:48.164799 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6526c6d2-d15c-4747-8cbd-a3cca208a260 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:48.165188 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6526c6d2-d15c-4747-8cbd-a3cca208a260 admin admin] PATH_INFO: `/v3/projects/baec172d59054da499387e85a4b8727e` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:48.166582 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6526c6d2-d15c-4747-8cbd-a3cca208a260 admin admin] RBAC: Authorizing `identity:delete_project(project_id=baec172d59054da499387e85a4b8727e)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:48.168907 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6526c6d2-d15c-4747-8cbd-a3cca208a260 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:48.216641 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:18:48.224854 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-6526c6d2-d15c-4747-8cbd-a3cca208a260 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'baec172d59054da499387e85a4b8727e', 'request_id': 'req-6526c6d2-d15c-4747-8cbd-a3cca208a260'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:48.231825 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-6526c6d2-d15c-4747-8cbd-a3cca208a260 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'baec172d59054da499387e85a4b8727e', 'request_id': 'req-6526c6d2-d15c-4747-8cbd-a3cca208a260'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:48.239350 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-6526c6d2-d15c-4747-8cbd-a3cca208a260 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'baec172d59054da499387e85a4b8727e', 'request_id': 'req-6526c6d2-d15c-4747-8cbd-a3cca208a260'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:48.251846 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-6526c6d2-d15c-4747-8cbd-a3cca208a260 admin admin] The token cache is being invalidate because project baec172d59054da499387e85a4b8727e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:18:48.252135 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-6526c6d2-d15c-4747-8cbd-a3cca208a260 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:48.252802 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-6526c6d2-d15c-4747-8cbd-a3cca208a260 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:48.254448 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1223/2465] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 9 12:18:48 2022] DELETE /identity/v3/projects/baec172d59054da499387e85a4b8727e => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:18:48.258345 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8f3afc-f403-40d0-a4f3-02a879c8aa14 service neutron] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:48.268828 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8f3afc-f403-40d0-a4f3-02a879c8aa14 service neutron] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:48.337896 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1a1db7-80a1-4eed-a520-2b38f8729cc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:48.338553 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1a1db7-80a1-4eed-a520-2b38f8729cc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:48.338821 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1a1db7-80a1-4eed-a520-2b38f8729cc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:48.339342 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1a1db7-80a1-4eed-a520-2b38f8729cc1 admin admin] PATH_INFO: `/v3/users/153a8689cf5644bf89bff141bab91d19` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:48.340279 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1a1db7-80a1-4eed-a520-2b38f8729cc1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=153a8689cf5644bf89bff141bab91d19)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:48.342200 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1a1db7-80a1-4eed-a520-2b38f8729cc1 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:48.397777 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-df1a1db7-80a1-4eed-a520-2b38f8729cc1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '153a8689cf5644bf89bff141bab91d19'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:48.407459 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-df1a1db7-80a1-4eed-a520-2b38f8729cc1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '153a8689cf5644bf89bff141bab91d19'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:48.420431 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-df1a1db7-80a1-4eed-a520-2b38f8729cc1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '153a8689cf5644bf89bff141bab91d19'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:48.428552 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-df1a1db7-80a1-4eed-a520-2b38f8729cc1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '153a8689cf5644bf89bff141bab91d19'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:48.429460 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-df1a1db7-80a1-4eed-a520-2b38f8729cc1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '153a8689cf5644bf89bff141bab91d19'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:48.438491 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1243/2466] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 9 12:18:48 2022] DELETE /identity/v3/users/153a8689cf5644bf89bff141bab91d19 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:18:48.458924 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6526c6d2-d15c-4747-8cbd-a3cca208a260 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:48.460733 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-6526c6d2-d15c-4747-8cbd-a3cca208a260 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:18:48.473919 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6526c6d2-d15c-4747-8cbd-a3cca208a260 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:48.581020 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7674380-8321-4e65-885c-9ecc6f995b1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:48.581838 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7674380-8321-4e65-885c-9ecc6f995b1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:48.582116 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7674380-8321-4e65-885c-9ecc6f995b1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:48.582525 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7674380-8321-4e65-885c-9ecc6f995b1e admin admin] PATH_INFO: `/v3/projects/7c59ff6b62944c88938bc02bf308f1b7` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:48.584130 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7674380-8321-4e65-885c-9ecc6f995b1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c59ff6b62944c88938bc02bf308f1b7)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:48.585958 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7674380-8321-4e65-885c-9ecc6f995b1e admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:48.644574 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-b7674380-8321-4e65-885c-9ecc6f995b1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c59ff6b62944c88938bc02bf308f1b7', 'request_id': 'req-b7674380-8321-4e65-885c-9ecc6f995b1e'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:48.654565 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-b7674380-8321-4e65-885c-9ecc6f995b1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c59ff6b62944c88938bc02bf308f1b7', 'request_id': 'req-b7674380-8321-4e65-885c-9ecc6f995b1e'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:48.656877 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1a1db7-80a1-4eed-a520-2b38f8729cc1 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:48.658466 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-df1a1db7-80a1-4eed-a520-2b38f8729cc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:18:48.670863 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-b7674380-8321-4e65-885c-9ecc6f995b1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c59ff6b62944c88938bc02bf308f1b7', 'request_id': 'req-b7674380-8321-4e65-885c-9ecc6f995b1e'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:48.674453 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1a1db7-80a1-4eed-a520-2b38f8729cc1 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:48.685522 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-b7674380-8321-4e65-885c-9ecc6f995b1e admin admin] The token cache is being invalidate because project 7c59ff6b62944c88938bc02bf308f1b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:18:48.685820 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-b7674380-8321-4e65-885c-9ecc6f995b1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:48.686495 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-b7674380-8321-4e65-885c-9ecc6f995b1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:48.688274 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1224/2467] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 9 12:18:48 2022] DELETE /identity/v3/projects/7c59ff6b62944c88938bc02bf308f1b7 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:18:48.692007 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7674380-8321-4e65-885c-9ecc6f995b1e admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:48.693496 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-b7674380-8321-4e65-885c-9ecc6f995b1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:18:48.744923 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae65694-79c0-44f8-9d4b-cc4d2a2f97b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:48.745573 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae65694-79c0-44f8-9d4b-cc4d2a2f97b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:48.745829 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae65694-79c0-44f8-9d4b-cc4d2a2f97b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:48.746096 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae65694-79c0-44f8-9d4b-cc4d2a2f97b3 admin admin] PATH_INFO: `/v3/projects/83258ee6a3a647c7aaf4eb729707ca1a` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:48.747268 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae65694-79c0-44f8-9d4b-cc4d2a2f97b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=83258ee6a3a647c7aaf4eb729707ca1a)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:48.749165 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae65694-79c0-44f8-9d4b-cc4d2a2f97b3 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:48.768060 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7674380-8321-4e65-885c-9ecc6f995b1e admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:48.772581 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:18:48.773088 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4bebcf-c314-49a1-ae43-f495c8b9f81e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:48.773714 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4bebcf-c314-49a1-ae43-f495c8b9f81e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:48.774011 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4bebcf-c314-49a1-ae43-f495c8b9f81e admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:48.774256 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4bebcf-c314-49a1-ae43-f495c8b9f81e admin admin] PATH_INFO: `/v3/users/09826346081749b4956661e7172ae5c4` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:48.775635 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4bebcf-c314-49a1-ae43-f495c8b9f81e admin admin] RBAC: Authorizing `identity:delete_user(user_id=09826346081749b4956661e7172ae5c4)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:48.777787 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4bebcf-c314-49a1-ae43-f495c8b9f81e admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:48.779665 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:18:48.809297 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-2ae65694-79c0-44f8-9d4b-cc4d2a2f97b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83258ee6a3a647c7aaf4eb729707ca1a', 'request_id': 'req-2ae65694-79c0-44f8-9d4b-cc4d2a2f97b3'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:48.816048 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-2ae65694-79c0-44f8-9d4b-cc4d2a2f97b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83258ee6a3a647c7aaf4eb729707ca1a', 'request_id': 'req-2ae65694-79c0-44f8-9d4b-cc4d2a2f97b3'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:48.825328 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-2ae65694-79c0-44f8-9d4b-cc4d2a2f97b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83258ee6a3a647c7aaf4eb729707ca1a', 'request_id': 'req-2ae65694-79c0-44f8-9d4b-cc4d2a2f97b3'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:48.840511 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-2ae65694-79c0-44f8-9d4b-cc4d2a2f97b3 admin admin] The token cache is being invalidate because project 83258ee6a3a647c7aaf4eb729707ca1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81591) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:18:48.840745 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-2ae65694-79c0-44f8-9d4b-cc4d2a2f97b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:48.841872 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-2ae65694-79c0-44f8-9d4b-cc4d2a2f97b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:48.843487 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1244/2468] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 9 12:18:48 2022] DELETE /identity/v3/projects/83258ee6a3a647c7aaf4eb729707ca1a => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:18:48.855124 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-af4bebcf-c314-49a1-ae43-f495c8b9f81e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09826346081749b4956661e7172ae5c4'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:48.865200 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-af4bebcf-c314-49a1-ae43-f495c8b9f81e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09826346081749b4956661e7172ae5c4'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:48.873954 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-af4bebcf-c314-49a1-ae43-f495c8b9f81e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09826346081749b4956661e7172ae5c4'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:48.883750 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-af4bebcf-c314-49a1-ae43-f495c8b9f81e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09826346081749b4956661e7172ae5c4'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:48.884640 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-af4bebcf-c314-49a1-ae43-f495c8b9f81e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09826346081749b4956661e7172ae5c4'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:48.891724 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1225/2469] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 9 12:18:48 2022] DELETE /identity/v3/users/09826346081749b4956661e7172ae5c4 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:18:49.187515 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae65694-79c0-44f8-9d4b-cc4d2a2f97b3 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:49.189183 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-2ae65694-79c0-44f8-9d4b-cc4d2a2f97b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:18:49.256133 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae65694-79c0-44f8-9d4b-cc4d2a2f97b3 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:49.311472 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356d7777-d29b-4228-bcb1-f5e13ec9195c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:49.312022 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356d7777-d29b-4228-bcb1-f5e13ec9195c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:49.312297 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356d7777-d29b-4228-bcb1-f5e13ec9195c admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:49.312563 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356d7777-d29b-4228-bcb1-f5e13ec9195c admin admin] PATH_INFO: `/v3/projects/1a9c4c78c0114854a50795dac49fbc49` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:49.313659 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356d7777-d29b-4228-bcb1-f5e13ec9195c admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a9c4c78c0114854a50795dac49fbc49)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:49.315618 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356d7777-d29b-4228-bcb1-f5e13ec9195c admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:49.360683 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-356d7777-d29b-4228-bcb1-f5e13ec9195c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a9c4c78c0114854a50795dac49fbc49', 'request_id': 'req-356d7777-d29b-4228-bcb1-f5e13ec9195c'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:49.366062 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-356d7777-d29b-4228-bcb1-f5e13ec9195c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a9c4c78c0114854a50795dac49fbc49', 'request_id': 'req-356d7777-d29b-4228-bcb1-f5e13ec9195c'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:49.367139 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:18:49.373738 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-356d7777-d29b-4228-bcb1-f5e13ec9195c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a9c4c78c0114854a50795dac49fbc49', 'request_id': 'req-356d7777-d29b-4228-bcb1-f5e13ec9195c'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:49.386754 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-356d7777-d29b-4228-bcb1-f5e13ec9195c admin admin] The token cache is being invalidate because project 1a9c4c78c0114854a50795dac49fbc49 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81591) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:18:49.387076 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-356d7777-d29b-4228-bcb1-f5e13ec9195c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:49.387711 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-356d7777-d29b-4228-bcb1-f5e13ec9195c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:49.389192 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1245/2470] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 9 12:18:49 2022] DELETE /identity/v3/projects/1a9c4c78c0114854a50795dac49fbc49 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:18:49.394217 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c8d3cb-1fb4-422d-aa14-8983076ef096 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:49.394636 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c8d3cb-1fb4-422d-aa14-8983076ef096 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:49.395013 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c8d3cb-1fb4-422d-aa14-8983076ef096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:49.415082 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-73c8d3cb-1fb4-422d-aa14-8983076ef096 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:18:49.416435 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-73c8d3cb-1fb4-422d-aa14-8983076ef096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:18:49.474848 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1226/2471] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:18:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:18:49.484052 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488da29b-3ba6-43a3-aec7-e74b2c90d3cf None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:49.484355 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488da29b-3ba6-43a3-aec7-e74b2c90d3cf None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:49.484651 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488da29b-3ba6-43a3-aec7-e74b2c90d3cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:49.504172 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-488da29b-3ba6-43a3-aec7-e74b2c90d3cf None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:18:49.505364 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-488da29b-3ba6-43a3-aec7-e74b2c90d3cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:18:49.513989 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1246/2472] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:18:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:18:49.518116 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c8d3cb-1fb4-422d-aa14-8983076ef096 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:49.528626 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c8d3cb-1fb4-422d-aa14-8983076ef096 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:49.533150 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d416ea7c-0568-4a2d-972d-9cc704f5c8c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:49.533777 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d416ea7c-0568-4a2d-972d-9cc704f5c8c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:49.534050 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d416ea7c-0568-4a2d-972d-9cc704f5c8c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:49.534317 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d416ea7c-0568-4a2d-972d-9cc704f5c8c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:49.535005 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d416ea7c-0568-4a2d-972d-9cc704f5c8c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:18:49.535439 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d416ea7c-0568-4a2d-972d-9cc704f5c8c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:49.537071 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d416ea7c-0568-4a2d-972d-9cc704f5c8c2 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:49.548578 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1227/2473] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 9 12:18:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:18:49.552764 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488da29b-3ba6-43a3-aec7-e74b2c90d3cf None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:49.562342 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488da29b-3ba6-43a3-aec7-e74b2c90d3cf None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:49.566926 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64660ed-a1f5-46a1-9bcb-5f2a04fb7e36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:49.567522 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64660ed-a1f5-46a1-9bcb-5f2a04fb7e36 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:49.567804 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64660ed-a1f5-46a1-9bcb-5f2a04fb7e36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:49.568070 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64660ed-a1f5-46a1-9bcb-5f2a04fb7e36 admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:49.568697 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64660ed-a1f5-46a1-9bcb-5f2a04fb7e36 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:49.570579 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64660ed-a1f5-46a1-9bcb-5f2a04fb7e36 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:49.600181 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1247/2474] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 9 12:18:49 2022] POST /identity/v3/projects => generated 387 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:18:49.604512 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d416ea7c-0568-4a2d-972d-9cc704f5c8c2 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:49.615295 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d416ea7c-0568-4a2d-972d-9cc704f5c8c2 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:49.684283 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709e8059-f2d2-4b79-b106-f953d4554a47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:49.684902 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709e8059-f2d2-4b79-b106-f953d4554a47 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:49.685180 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709e8059-f2d2-4b79-b106-f953d4554a47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:49.685478 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709e8059-f2d2-4b79-b106-f953d4554a47 admin admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:49.686132 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709e8059-f2d2-4b79-b106-f953d4554a47 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:49.688222 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709e8059-f2d2-4b79-b106-f953d4554a47 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:49.712631 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:18:49.716284 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1228/2475] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:18:49 2022] POST /identity/v3/users => generated 294 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:18:49.717778 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64660ed-a1f5-46a1-9bcb-5f2a04fb7e36 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:49.729454 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64660ed-a1f5-46a1-9bcb-5f2a04fb7e36 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:49.734146 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaad40a-7eb1-4306-8872-d1d705e9f61e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:49.734753 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaad40a-7eb1-4306-8872-d1d705e9f61e admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:49.735066 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaad40a-7eb1-4306-8872-d1d705e9f61e admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:49.735341 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaad40a-7eb1-4306-8872-d1d705e9f61e admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:49.735988 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaad40a-7eb1-4306-8872-d1d705e9f61e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:49.737589 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaad40a-7eb1-4306-8872-d1d705e9f61e admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:49.752785 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1248/2476] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:18:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:18:49.753647 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709e8059-f2d2-4b79-b106-f953d4554a47 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:49.769387 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709e8059-f2d2-4b79-b106-f953d4554a47 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:49.774191 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1755390-7f42-44a7-9cd8-ff7b00d82174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:49.774858 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1755390-7f42-44a7-9cd8-ff7b00d82174 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:49.775290 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1755390-7f42-44a7-9cd8-ff7b00d82174 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:49.775560 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1755390-7f42-44a7-9cd8-ff7b00d82174 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:49.776199 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1755390-7f42-44a7-9cd8-ff7b00d82174 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:49.777830 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1755390-7f42-44a7-9cd8-ff7b00d82174 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:49.788501 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1229/2477] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:18:49 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:18:49.792652 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaad40a-7eb1-4306-8872-d1d705e9f61e admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:49.802479 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaad40a-7eb1-4306-8872-d1d705e9f61e admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:49.808094 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a52a258-adfa-4012-8ee0-0ea8a5df99b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:49.808734 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a52a258-adfa-4012-8ee0-0ea8a5df99b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:49.809005 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a52a258-adfa-4012-8ee0-0ea8a5df99b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:49.809270 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a52a258-adfa-4012-8ee0-0ea8a5df99b7 admin admin] PATH_INFO: `/v3/projects/57cee0df4201475fbdbb6b5b6f976a4a/users/6dd4b25614244dc980e2a6731b1214b6/roles/eecd2f160dd844f5906f3d345780c501` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:49.824392 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a52a258-adfa-4012-8ee0-0ea8a5df99b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57cee0df4201475fbdbb6b5b6f976a4a, user_id=6dd4b25614244dc980e2a6731b1214b6, role_id=eecd2f160dd844f5906f3d345780c501)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:49.826974 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a52a258-adfa-4012-8ee0-0ea8a5df99b7 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:49.840329 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1249/2478] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:18:49 2022] PUT /identity/v3/projects/57cee0df4201475fbdbb6b5b6f976a4a/users/6dd4b25614244dc980e2a6731b1214b6/roles/eecd2f160dd844f5906f3d345780c501 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:18:49.850658 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082f4132-b032-45c0-a6d2-ff44eb08b541 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:49.850990 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082f4132-b032-45c0-a6d2-ff44eb08b541 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:49.851261 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082f4132-b032-45c0-a6d2-ff44eb08b541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:49.878685 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-082f4132-b032-45c0-a6d2-ff44eb08b541 None None] MFA Rules not processed for user `6dd4b25614244dc980e2a6731b1214b6`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:18:49.879933 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-082f4132-b032-45c0-a6d2-ff44eb08b541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:18:49.943734 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1230/2479] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:18:49 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:18:49.949081 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a52a258-adfa-4012-8ee0-0ea8a5df99b7 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:50.008714 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a52a258-adfa-4012-8ee0-0ea8a5df99b7 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:50.014131 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f1b62b-0b10-4cd4-8d17-6d6c3a39747e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:50.014723 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f1b62b-0b10-4cd4-8d17-6d6c3a39747e admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:50.015028 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f1b62b-0b10-4cd4-8d17-6d6c3a39747e admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:50.015298 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f1b62b-0b10-4cd4-8d17-6d6c3a39747e admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:50.015905 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f1b62b-0b10-4cd4-8d17-6d6c3a39747e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:50.017788 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f1b62b-0b10-4cd4-8d17-6d6c3a39747e admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:50.044117 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1250/2480] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 9 12:18:49 2022] POST /identity/v3/projects => generated 385 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:18:50.048508 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082f4132-b032-45c0-a6d2-ff44eb08b541 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:50.060307 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082f4132-b032-45c0-a6d2-ff44eb08b541 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:50.120010 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e4080a-a91a-4ac7-9490-275a99deca15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:50.120630 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e4080a-a91a-4ac7-9490-275a99deca15 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:50.120911 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e4080a-a91a-4ac7-9490-275a99deca15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:50.121184 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e4080a-a91a-4ac7-9490-275a99deca15 admin admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:50.121829 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e4080a-a91a-4ac7-9490-275a99deca15 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:50.123818 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e4080a-a91a-4ac7-9490-275a99deca15 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:50.148809 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1231/2481] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:18:50 2022] POST /identity/v3/users => generated 293 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:18:50.150052 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f1b62b-0b10-4cd4-8d17-6d6c3a39747e admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:50.160410 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f1b62b-0b10-4cd4-8d17-6d6c3a39747e admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:50.165564 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a8d8ff-28f8-40fa-b5b5-8ab5a28b0b94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:50.166163 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a8d8ff-28f8-40fa-b5b5-8ab5a28b0b94 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:50.166439 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a8d8ff-28f8-40fa-b5b5-8ab5a28b0b94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:50.166709 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a8d8ff-28f8-40fa-b5b5-8ab5a28b0b94 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:50.167497 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a8d8ff-28f8-40fa-b5b5-8ab5a28b0b94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:50.169077 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a8d8ff-28f8-40fa-b5b5-8ab5a28b0b94 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:50.180312 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1251/2482] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:18:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 12:18:50.184943 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e4080a-a91a-4ac7-9490-275a99deca15 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:50.195668 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e4080a-a91a-4ac7-9490-275a99deca15 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:50.200789 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d446744b-b435-4845-9cfd-f3e0ac817e15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:50.201392 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d446744b-b435-4845-9cfd-f3e0ac817e15 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:50.201673 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d446744b-b435-4845-9cfd-f3e0ac817e15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:50.201947 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d446744b-b435-4845-9cfd-f3e0ac817e15 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:50.202586 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d446744b-b435-4845-9cfd-f3e0ac817e15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:50.204223 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d446744b-b435-4845-9cfd-f3e0ac817e15 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:50.215262 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1232/2483] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:18:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:18:50.219431 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a8d8ff-28f8-40fa-b5b5-8ab5a28b0b94 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:50.230771 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a8d8ff-28f8-40fa-b5b5-8ab5a28b0b94 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:50.235409 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4825be52-81ed-4b23-8863-b0e97c42f9e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:50.236060 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4825be52-81ed-4b23-8863-b0e97c42f9e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:50.236338 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4825be52-81ed-4b23-8863-b0e97c42f9e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:50.236641 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4825be52-81ed-4b23-8863-b0e97c42f9e6 admin admin] PATH_INFO: `/v3/projects/7a47679fef724e53b74eef41f6849220/users/46af643bc9d54da58d17fab03087824d/roles/eecd2f160dd844f5906f3d345780c501` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:50.252234 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4825be52-81ed-4b23-8863-b0e97c42f9e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a47679fef724e53b74eef41f6849220, user_id=46af643bc9d54da58d17fab03087824d, role_id=eecd2f160dd844f5906f3d345780c501)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:50.254890 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4825be52-81ed-4b23-8863-b0e97c42f9e6 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:50.268678 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1252/2484] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:18:50 2022] PUT /identity/v3/projects/7a47679fef724e53b74eef41f6849220/users/46af643bc9d54da58d17fab03087824d/roles/eecd2f160dd844f5906f3d345780c501 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:18:50.280030 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fcb25c-af67-4ce2-b557-6fb25ecabbca None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:50.280353 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fcb25c-af67-4ce2-b557-6fb25ecabbca None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:50.280626 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fcb25c-af67-4ce2-b557-6fb25ecabbca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:50.309799 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-82fcb25c-af67-4ce2-b557-6fb25ecabbca None None] MFA Rules not processed for user `46af643bc9d54da58d17fab03087824d`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:18:50.311044 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-82fcb25c-af67-4ce2-b557-6fb25ecabbca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:18:50.353379 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:18:50.377070 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1233/2485] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:18:50 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:18:50.381879 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4825be52-81ed-4b23-8863-b0e97c42f9e6 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:50.418002 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:18:50.447359 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4825be52-81ed-4b23-8863-b0e97c42f9e6 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:50.452203 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f629b6c6-23c3-42ea-b84d-b339568fbf20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:50.452814 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f629b6c6-23c3-42ea-b84d-b339568fbf20 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:50.453096 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f629b6c6-23c3-42ea-b84d-b339568fbf20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:50.453363 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f629b6c6-23c3-42ea-b84d-b339568fbf20 admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:50.454004 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f629b6c6-23c3-42ea-b84d-b339568fbf20 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:50.456003 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f629b6c6-23c3-42ea-b84d-b339568fbf20 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:50.485045 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1253/2486] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 9 12:18:50 2022] POST /identity/v3/projects => generated 387 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:18:50.489137 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fcb25c-af67-4ce2-b557-6fb25ecabbca None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:50.500367 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fcb25c-af67-4ce2-b557-6fb25ecabbca None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:50.558863 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d7502f-88e3-4fcb-9f50-b5d16561a936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:50.559401 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d7502f-88e3-4fcb-9f50-b5d16561a936 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:50.559675 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d7502f-88e3-4fcb-9f50-b5d16561a936 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:50.560112 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d7502f-88e3-4fcb-9f50-b5d16561a936 admin admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:50.560593 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d7502f-88e3-4fcb-9f50-b5d16561a936 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:50.562536 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d7502f-88e3-4fcb-9f50-b5d16561a936 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:50.589054 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1234/2487] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:18:50 2022] POST /identity/v3/users => generated 300 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:18:50.590228 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f629b6c6-23c3-42ea-b84d-b339568fbf20 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:50.601097 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f629b6c6-23c3-42ea-b84d-b339568fbf20 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:50.606756 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca4cfeb-ac2f-4119-bf0b-0d253f9ade00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:50.607403 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca4cfeb-ac2f-4119-bf0b-0d253f9ade00 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:50.607686 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca4cfeb-ac2f-4119-bf0b-0d253f9ade00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:50.607960 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca4cfeb-ac2f-4119-bf0b-0d253f9ade00 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:50.608603 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca4cfeb-ac2f-4119-bf0b-0d253f9ade00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:50.610191 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca4cfeb-ac2f-4119-bf0b-0d253f9ade00 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:50.621117 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1254/2488] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:18:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:18:50.629543 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d7502f-88e3-4fcb-9f50-b5d16561a936 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:50.636461 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d7502f-88e3-4fcb-9f50-b5d16561a936 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:50.645886 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c839be2f-ce01-4d48-8447-0d599278dd07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:50.646703 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c839be2f-ce01-4d48-8447-0d599278dd07 admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:50.647127 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c839be2f-ce01-4d48-8447-0d599278dd07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:50.647985 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c839be2f-ce01-4d48-8447-0d599278dd07 admin admin] PATH_INFO: `/v3/projects/8ef382d021994b6eb92b2536a8c2712f/users/0d2a809ef1f74a368e44dd68cb9896e2/roles/3f2ca33902a04b6dbc2ef7d06d7289d3` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:50.666021 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c839be2f-ce01-4d48-8447-0d599278dd07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ef382d021994b6eb92b2536a8c2712f, user_id=0d2a809ef1f74a368e44dd68cb9896e2, role_id=3f2ca33902a04b6dbc2ef7d06d7289d3)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:50.668822 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c839be2f-ce01-4d48-8447-0d599278dd07 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:50.682516 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1235/2489] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:18:50 2022] PUT /identity/v3/projects/8ef382d021994b6eb92b2536a8c2712f/users/0d2a809ef1f74a368e44dd68cb9896e2/roles/3f2ca33902a04b6dbc2ef7d06d7289d3 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:18:50.686627 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca4cfeb-ac2f-4119-bf0b-0d253f9ade00 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:50.696968 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca4cfeb-ac2f-4119-bf0b-0d253f9ade00 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:50.755791 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bb8346-0352-41df-af66-97dc1f69d416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:50.756394 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bb8346-0352-41df-af66-97dc1f69d416 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:50.756670 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bb8346-0352-41df-af66-97dc1f69d416 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:50.756941 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bb8346-0352-41df-af66-97dc1f69d416 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:50.757723 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bb8346-0352-41df-af66-97dc1f69d416 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:50.760143 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bb8346-0352-41df-af66-97dc1f69d416 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:50.773722 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1255/2490] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:18:50 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:18:50.778193 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c839be2f-ce01-4d48-8447-0d599278dd07 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:50.792946 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c839be2f-ce01-4d48-8447-0d599278dd07 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:50.799193 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42322f69-c9a9-4036-b3bb-1ffd26732f28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:50.799868 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42322f69-c9a9-4036-b3bb-1ffd26732f28 admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:50.800150 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42322f69-c9a9-4036-b3bb-1ffd26732f28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:50.800797 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42322f69-c9a9-4036-b3bb-1ffd26732f28 admin admin] PATH_INFO: `/v3/projects/8ef382d021994b6eb92b2536a8c2712f/users/0d2a809ef1f74a368e44dd68cb9896e2/roles/3f2ca33902a04b6dbc2ef7d06d7289d3` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:50.802384 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42322f69-c9a9-4036-b3bb-1ffd26732f28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ef382d021994b6eb92b2536a8c2712f, user_id=0d2a809ef1f74a368e44dd68cb9896e2, role_id=3f2ca33902a04b6dbc2ef7d06d7289d3)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:50.806236 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42322f69-c9a9-4036-b3bb-1ffd26732f28 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:50.819504 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1236/2491] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:18:50 2022] PUT /identity/v3/projects/8ef382d021994b6eb92b2536a8c2712f/users/0d2a809ef1f74a368e44dd68cb9896e2/roles/3f2ca33902a04b6dbc2ef7d06d7289d3 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:18:50.832604 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59769028-a333-4f10-96bb-9295ad238ce5 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:50.833141 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59769028-a333-4f10-96bb-9295ad238ce5 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:50.833141 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59769028-a333-4f10-96bb-9295ad238ce5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:50.861611 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-59769028-a333-4f10-96bb-9295ad238ce5 None None] MFA Rules not processed for user `0d2a809ef1f74a368e44dd68cb9896e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:18:50.862744 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-59769028-a333-4f10-96bb-9295ad238ce5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:18:50.934307 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1256/2492] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:18:50 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:18:50.944957 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42322f69-c9a9-4036-b3bb-1ffd26732f28 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:50.947028 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-42322f69-c9a9-4036-b3bb-1ffd26732f28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:18:51.004317 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42322f69-c9a9-4036-b3bb-1ffd26732f28 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:51.008029 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc077fd-ee5a-4728-a204-f81ff3138363 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535c258100df40b58a5ee44b2c21d621', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:51.008654 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc077fd-ee5a-4728-a204-f81ff3138363 service cinder] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:51.008930 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc077fd-ee5a-4728-a204-f81ff3138363 service cinder] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:51.009208 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc077fd-ee5a-4728-a204-f81ff3138363 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:51.018166 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc077fd-ee5a-4728-a204-f81ff3138363 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:51.019615 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc077fd-ee5a-4728-a204-f81ff3138363 service cinder] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:51.029031 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1237/2493] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:18:50 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:18:51.532796 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59769028-a333-4f10-96bb-9295ad238ce5 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:51.534269 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-59769028-a333-4f10-96bb-9295ad238ce5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:18:51.545934 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59769028-a333-4f10-96bb-9295ad238ce5 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:51.549563 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a115af-9de5-4567-82eb-f26d9806e201 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535c258100df40b58a5ee44b2c21d621', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:51.550247 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a115af-9de5-4567-82eb-f26d9806e201 service cinder] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:51.550610 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a115af-9de5-4567-82eb-f26d9806e201 service cinder] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:51.551194 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a115af-9de5-4567-82eb-f26d9806e201 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:51.582492 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:18:51.615461 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a115af-9de5-4567-82eb-f26d9806e201 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:51.616911 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a115af-9de5-4567-82eb-f26d9806e201 service cinder] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:51.626580 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1257/2494] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:18:51 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:18:51.692720 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031b534d-610e-4239-81bd-3e9421ffb6d8 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:51.693129 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031b534d-610e-4239-81bd-3e9421ffb6d8 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:51.693451 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031b534d-610e-4239-81bd-3e9421ffb6d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:51.715064 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-031b534d-610e-4239-81bd-3e9421ffb6d8 None None] MFA Rules not processed for user `e368b4e668c54e25bff80a8e8a8c3588`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:18:51.716274 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-031b534d-610e-4239-81bd-3e9421ffb6d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:18:51.750185 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:18:51.775473 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1238/2495] 10.222.0.49 () {58 vars in 1009 bytes} [Fri Sep 9 12:18:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:18:51.789651 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a115af-9de5-4567-82eb-f26d9806e201 service cinder] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:51.792071 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-75a115af-9de5-4567-82eb-f26d9806e201 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:18:51.871425 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a115af-9de5-4567-82eb-f26d9806e201 service cinder] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:51.883334 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3000c0cc-8d85-4baa-a7a4-22278dd95484 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e652ba827849b5a3fc1213d04a1b76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:51.884068 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3000c0cc-8d85-4baa-a7a4-22278dd95484 service swift] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:51.884449 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3000c0cc-8d85-4baa-a7a4-22278dd95484 service swift] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:51.884829 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3000c0cc-8d85-4baa-a7a4-22278dd95484 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:51.901977 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3000c0cc-8d85-4baa-a7a4-22278dd95484 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:51.903655 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3000c0cc-8d85-4baa-a7a4-22278dd95484 service swift] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:51.912273 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1258/2496] 10.222.0.49 () {60 vars in 1384 bytes} [Fri Sep 9 12:18:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 09 12:18:55.903960 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031b534d-610e-4239-81bd-3e9421ffb6d8 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:55.934958 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031b534d-610e-4239-81bd-3e9421ffb6d8 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:55.939337 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7337c7-5748-44e2-847b-fffc372dbd90 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535c258100df40b58a5ee44b2c21d621', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:55.940152 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7337c7-5748-44e2-847b-fffc372dbd90 service cinder] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:55.940605 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7337c7-5748-44e2-847b-fffc372dbd90 service cinder] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:55.940875 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7337c7-5748-44e2-847b-fffc372dbd90 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:56.017990 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7337c7-5748-44e2-847b-fffc372dbd90 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:56.019501 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7337c7-5748-44e2-847b-fffc372dbd90 service cinder] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:56.037084 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1239/2497] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:18:55 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:18:58.396119 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3000c0cc-8d85-4baa-a7a4-22278dd95484 service swift] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:58.474701 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3000c0cc-8d85-4baa-a7a4-22278dd95484 service swift] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:58.480479 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23ffb69-1786-4ed2-bfea-2ad059668481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:58.481273 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23ffb69-1786-4ed2-bfea-2ad059668481 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:58.481662 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23ffb69-1786-4ed2-bfea-2ad059668481 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:58.482040 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23ffb69-1786-4ed2-bfea-2ad059668481 admin admin] PATH_INFO: `/v3/users/6dd4b25614244dc980e2a6731b1214b6` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:58.483319 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23ffb69-1786-4ed2-bfea-2ad059668481 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6dd4b25614244dc980e2a6731b1214b6)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:58.485369 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23ffb69-1786-4ed2-bfea-2ad059668481 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:58.538113 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-f23ffb69-1786-4ed2-bfea-2ad059668481 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6dd4b25614244dc980e2a6731b1214b6'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:58.545926 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-f23ffb69-1786-4ed2-bfea-2ad059668481 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6dd4b25614244dc980e2a6731b1214b6'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:58.547846 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:18:58.563550 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-f23ffb69-1786-4ed2-bfea-2ad059668481 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6dd4b25614244dc980e2a6731b1214b6'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:58.573500 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-f23ffb69-1786-4ed2-bfea-2ad059668481 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6dd4b25614244dc980e2a6731b1214b6'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:58.574413 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-f23ffb69-1786-4ed2-bfea-2ad059668481 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6dd4b25614244dc980e2a6731b1214b6'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:58.581191 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1259/2498] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 9 12:18:58 2022] DELETE /identity/v3/users/6dd4b25614244dc980e2a6731b1214b6 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:18:58.587364 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c190e2af-527f-437e-888d-7c34852e4d9d None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:58.587897 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c190e2af-527f-437e-888d-7c34852e4d9d None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:58.588208 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c190e2af-527f-437e-888d-7c34852e4d9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:58.608038 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-c190e2af-527f-437e-888d-7c34852e4d9d None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:18:58.609418 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-c190e2af-527f-437e-888d-7c34852e4d9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:18:58.683968 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1240/2499] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:18:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:18:58.693804 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23ffb69-1786-4ed2-bfea-2ad059668481 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:58.695414 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-f23ffb69-1786-4ed2-bfea-2ad059668481 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:18:58.772577 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23ffb69-1786-4ed2-bfea-2ad059668481 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:58.777177 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6695cd5b-0e4c-4249-8c52-3489418098be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:58.777861 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6695cd5b-0e4c-4249-8c52-3489418098be service neutron] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:58.778225 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6695cd5b-0e4c-4249-8c52-3489418098be service neutron] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:58.778563 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6695cd5b-0e4c-4249-8c52-3489418098be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:58.791299 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6695cd5b-0e4c-4249-8c52-3489418098be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:58.792877 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6695cd5b-0e4c-4249-8c52-3489418098be service neutron] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:58.815008 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1260/2500] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:18:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:18:59.120939 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c190e2af-527f-437e-888d-7c34852e4d9d None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:59.123497 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-c190e2af-527f-437e-888d-7c34852e4d9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:18:59.140009 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c190e2af-527f-437e-888d-7c34852e4d9d None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:59.145229 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed62d88b-644c-4d64-bfa0-ba973f5470e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:59.145959 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed62d88b-644c-4d64-bfa0-ba973f5470e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:59.146309 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed62d88b-644c-4d64-bfa0-ba973f5470e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:59.146653 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed62d88b-644c-4d64-bfa0-ba973f5470e4 admin admin] PATH_INFO: `/v3/projects/57cee0df4201475fbdbb6b5b6f976a4a` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:59.148224 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed62d88b-644c-4d64-bfa0-ba973f5470e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=57cee0df4201475fbdbb6b5b6f976a4a)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:59.150100 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed62d88b-644c-4d64-bfa0-ba973f5470e4 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:59.210854 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-ed62d88b-644c-4d64-bfa0-ba973f5470e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57cee0df4201475fbdbb6b5b6f976a4a', 'request_id': 'req-ed62d88b-644c-4d64-bfa0-ba973f5470e4'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:59.217847 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-ed62d88b-644c-4d64-bfa0-ba973f5470e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57cee0df4201475fbdbb6b5b6f976a4a', 'request_id': 'req-ed62d88b-644c-4d64-bfa0-ba973f5470e4'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:59.229955 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-ed62d88b-644c-4d64-bfa0-ba973f5470e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57cee0df4201475fbdbb6b5b6f976a4a', 'request_id': 'req-ed62d88b-644c-4d64-bfa0-ba973f5470e4'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:59.244802 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-ed62d88b-644c-4d64-bfa0-ba973f5470e4 admin admin] The token cache is being invalidate because project 57cee0df4201475fbdbb6b5b6f976a4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:18:59.245258 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-ed62d88b-644c-4d64-bfa0-ba973f5470e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:59.246442 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-ed62d88b-644c-4d64-bfa0-ba973f5470e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:59.247984 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1241/2501] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 9 12:18:59 2022] DELETE /identity/v3/projects/57cee0df4201475fbdbb6b5b6f976a4a => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:18:59.251927 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6695cd5b-0e4c-4249-8c52-3489418098be service neutron] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:59.261870 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6695cd5b-0e4c-4249-8c52-3489418098be service neutron] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:59.329737 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fea775-eb5d-4784-9de3-86f38f90e35f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:59.330279 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fea775-eb5d-4784-9de3-86f38f90e35f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:59.330555 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fea775-eb5d-4784-9de3-86f38f90e35f admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:59.330822 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fea775-eb5d-4784-9de3-86f38f90e35f admin admin] PATH_INFO: `/v3/users/46af643bc9d54da58d17fab03087824d` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:59.331988 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fea775-eb5d-4784-9de3-86f38f90e35f admin admin] RBAC: Authorizing `identity:delete_user(user_id=46af643bc9d54da58d17fab03087824d)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:59.333928 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fea775-eb5d-4784-9de3-86f38f90e35f admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:59.399321 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-f5fea775-eb5d-4784-9de3-86f38f90e35f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46af643bc9d54da58d17fab03087824d'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:59.418099 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-f5fea775-eb5d-4784-9de3-86f38f90e35f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46af643bc9d54da58d17fab03087824d'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:59.432140 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-f5fea775-eb5d-4784-9de3-86f38f90e35f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46af643bc9d54da58d17fab03087824d'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:59.444495 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-f5fea775-eb5d-4784-9de3-86f38f90e35f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46af643bc9d54da58d17fab03087824d'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:59.445651 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-f5fea775-eb5d-4784-9de3-86f38f90e35f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46af643bc9d54da58d17fab03087824d'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:59.447897 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:18:59.454976 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1261/2502] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 9 12:18:59 2022] DELETE /identity/v3/users/46af643bc9d54da58d17fab03087824d => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:18:59.818080 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed62d88b-644c-4d64-bfa0-ba973f5470e4 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:18:59.819680 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-ed62d88b-644c-4d64-bfa0-ba973f5470e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:18:59.843007 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:18:59.849632 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed62d88b-644c-4d64-bfa0-ba973f5470e4 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:18:59.923430 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369cca20-ad87-457f-8d66-d167748d3463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:18:59.924399 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369cca20-ad87-457f-8d66-d167748d3463 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:18:59.924850 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369cca20-ad87-457f-8d66-d167748d3463 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:18:59.925434 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369cca20-ad87-457f-8d66-d167748d3463 admin admin] PATH_INFO: `/v3/projects/7a47679fef724e53b74eef41f6849220` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:18:59.926799 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369cca20-ad87-457f-8d66-d167748d3463 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a47679fef724e53b74eef41f6849220)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:18:59.929100 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369cca20-ad87-457f-8d66-d167748d3463 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:18:59.987400 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-369cca20-ad87-457f-8d66-d167748d3463 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a47679fef724e53b74eef41f6849220', 'request_id': 'req-369cca20-ad87-457f-8d66-d167748d3463'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:18:59.994570 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-369cca20-ad87-457f-8d66-d167748d3463 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a47679fef724e53b74eef41f6849220', 'request_id': 'req-369cca20-ad87-457f-8d66-d167748d3463'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:00.008546 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-369cca20-ad87-457f-8d66-d167748d3463 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a47679fef724e53b74eef41f6849220', 'request_id': 'req-369cca20-ad87-457f-8d66-d167748d3463'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:00.026413 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-369cca20-ad87-457f-8d66-d167748d3463 admin admin] The token cache is being invalidate because project 7a47679fef724e53b74eef41f6849220 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:19:00.026734 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-369cca20-ad87-457f-8d66-d167748d3463 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:00.027646 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-369cca20-ad87-457f-8d66-d167748d3463 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:00.029199 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1242/2503] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 9 12:18:59 2022] DELETE /identity/v3/projects/7a47679fef724e53b74eef41f6849220 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:19:00.033336 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fea775-eb5d-4784-9de3-86f38f90e35f admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:00.036873 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-f5fea775-eb5d-4784-9de3-86f38f90e35f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:19:00.057034 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fea775-eb5d-4784-9de3-86f38f90e35f admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:00.145058 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50811d50-e4af-453d-a430-c6aeb202e2e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:00.145712 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50811d50-e4af-453d-a430-c6aeb202e2e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:00.145987 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50811d50-e4af-453d-a430-c6aeb202e2e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:00.146256 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50811d50-e4af-453d-a430-c6aeb202e2e1 admin admin] PATH_INFO: `/v3/users/0d2a809ef1f74a368e44dd68cb9896e2` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:00.147415 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50811d50-e4af-453d-a430-c6aeb202e2e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d2a809ef1f74a368e44dd68cb9896e2)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:00.150836 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50811d50-e4af-453d-a430-c6aeb202e2e1 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:00.221224 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-50811d50-e4af-453d-a430-c6aeb202e2e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d2a809ef1f74a368e44dd68cb9896e2'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:00.229752 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-50811d50-e4af-453d-a430-c6aeb202e2e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d2a809ef1f74a368e44dd68cb9896e2'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:00.244071 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-50811d50-e4af-453d-a430-c6aeb202e2e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d2a809ef1f74a368e44dd68cb9896e2'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:00.257925 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-50811d50-e4af-453d-a430-c6aeb202e2e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d2a809ef1f74a368e44dd68cb9896e2'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:00.259624 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-50811d50-e4af-453d-a430-c6aeb202e2e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d2a809ef1f74a368e44dd68cb9896e2'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:00.268787 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1262/2504] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 9 12:19:00 2022] DELETE /identity/v3/users/0d2a809ef1f74a368e44dd68cb9896e2 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:19:00.632099 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369cca20-ad87-457f-8d66-d167748d3463 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:00.633912 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-369cca20-ad87-457f-8d66-d167748d3463 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:19:00.665180 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369cca20-ad87-457f-8d66-d167748d3463 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:00.784717 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9ce14a-0c4d-4085-b49e-8f2a4016c11f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:00.785507 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9ce14a-0c4d-4085-b49e-8f2a4016c11f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:00.785785 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9ce14a-0c4d-4085-b49e-8f2a4016c11f admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:00.786051 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9ce14a-0c4d-4085-b49e-8f2a4016c11f admin admin] PATH_INFO: `/v3/projects/8ef382d021994b6eb92b2536a8c2712f` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:00.787222 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9ce14a-0c4d-4085-b49e-8f2a4016c11f admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ef382d021994b6eb92b2536a8c2712f)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:00.789149 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9ce14a-0c4d-4085-b49e-8f2a4016c11f admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:00.845633 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:19:00.858488 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-ed9ce14a-0c4d-4085-b49e-8f2a4016c11f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ef382d021994b6eb92b2536a8c2712f', 'request_id': 'req-ed9ce14a-0c4d-4085-b49e-8f2a4016c11f'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:00.866509 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-ed9ce14a-0c4d-4085-b49e-8f2a4016c11f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ef382d021994b6eb92b2536a8c2712f', 'request_id': 'req-ed9ce14a-0c4d-4085-b49e-8f2a4016c11f'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:00.877054 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-ed9ce14a-0c4d-4085-b49e-8f2a4016c11f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ef382d021994b6eb92b2536a8c2712f', 'request_id': 'req-ed9ce14a-0c4d-4085-b49e-8f2a4016c11f'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:00.893094 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-ed9ce14a-0c4d-4085-b49e-8f2a4016c11f admin admin] The token cache is being invalidate because project 8ef382d021994b6eb92b2536a8c2712f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:19:00.893386 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-ed9ce14a-0c4d-4085-b49e-8f2a4016c11f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:00.894043 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-ed9ce14a-0c4d-4085-b49e-8f2a4016c11f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:00.895846 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1243/2505] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 9 12:19:00 2022] DELETE /identity/v3/projects/8ef382d021994b6eb92b2536a8c2712f => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:19:00.902022 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940184df-3359-4cd6-9dc6-42c3527e9508 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:00.902549 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940184df-3359-4cd6-9dc6-42c3527e9508 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:00.902549 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940184df-3359-4cd6-9dc6-42c3527e9508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:00.922727 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-940184df-3359-4cd6-9dc6-42c3527e9508 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:19:00.924021 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-940184df-3359-4cd6-9dc6-42c3527e9508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:19:00.997565 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1263/2506] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:19:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:19:01.006218 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4ed038-eec2-495e-9877-f761605f7856 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:01.006503 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4ed038-eec2-495e-9877-f761605f7856 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:01.006777 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4ed038-eec2-495e-9877-f761605f7856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:01.028687 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-9a4ed038-eec2-495e-9877-f761605f7856 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:19:01.030062 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-9a4ed038-eec2-495e-9877-f761605f7856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:19:01.040388 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1244/2507] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:19:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:19:01.044897 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940184df-3359-4cd6-9dc6-42c3527e9508 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:01.057045 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940184df-3359-4cd6-9dc6-42c3527e9508 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:01.062437 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6c4e61-7328-47f4-be7c-fef821b47ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:01.063164 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6c4e61-7328-47f4-be7c-fef821b47ce8 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:01.063566 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6c4e61-7328-47f4-be7c-fef821b47ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:01.063902 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6c4e61-7328-47f4-be7c-fef821b47ce8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:01.064611 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6c4e61-7328-47f4-be7c-fef821b47ce8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:19:01.064972 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6c4e61-7328-47f4-be7c-fef821b47ce8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:01.066632 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6c4e61-7328-47f4-be7c-fef821b47ce8 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:01.080848 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1264/2508] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 9 12:19:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:19:01.085481 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4ed038-eec2-495e-9877-f761605f7856 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:01.095463 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4ed038-eec2-495e-9877-f761605f7856 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:01.100708 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0b3650-969e-4078-b22b-650d05970995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:01.101444 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0b3650-969e-4078-b22b-650d05970995 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:01.101947 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0b3650-969e-4078-b22b-650d05970995 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:01.102409 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0b3650-969e-4078-b22b-650d05970995 admin admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:01.103293 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0b3650-969e-4078-b22b-650d05970995 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:01.105581 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0b3650-969e-4078-b22b-650d05970995 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:01.136986 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1245/2509] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 9 12:19:01 2022] POST /identity/v3/projects => generated 365 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:19:01.140865 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6c4e61-7328-47f4-be7c-fef821b47ce8 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:01.147313 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:19:01.151604 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6c4e61-7328-47f4-be7c-fef821b47ce8 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:01.213599 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d3eece-44d7-4373-831d-53c55508dcac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:01.214225 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d3eece-44d7-4373-831d-53c55508dcac admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:01.214502 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d3eece-44d7-4373-831d-53c55508dcac admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:01.214768 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d3eece-44d7-4373-831d-53c55508dcac admin admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:01.215594 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d3eece-44d7-4373-831d-53c55508dcac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:01.217660 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d3eece-44d7-4373-831d-53c55508dcac admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:01.242223 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1265/2510] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:19:01 2022] POST /identity/v3/users => generated 283 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:19:01.247463 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0b3650-969e-4078-b22b-650d05970995 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:01.259091 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0b3650-969e-4078-b22b-650d05970995 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:01.269978 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f6d617-b266-4f9b-b0db-7d4531a88511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:01.270635 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f6d617-b266-4f9b-b0db-7d4531a88511 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:01.271018 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f6d617-b266-4f9b-b0db-7d4531a88511 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:01.271294 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f6d617-b266-4f9b-b0db-7d4531a88511 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:01.271943 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f6d617-b266-4f9b-b0db-7d4531a88511 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:01.273603 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f6d617-b266-4f9b-b0db-7d4531a88511 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:01.284606 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1246/2511] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:19:01 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:19:01.288459 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d3eece-44d7-4373-831d-53c55508dcac admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:01.299556 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d3eece-44d7-4373-831d-53c55508dcac admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:01.304455 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa39937e-cb8d-4c29-88c5-1a129a56c29c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:01.305612 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa39937e-cb8d-4c29-88c5-1a129a56c29c admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:01.306138 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa39937e-cb8d-4c29-88c5-1a129a56c29c admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:01.306550 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa39937e-cb8d-4c29-88c5-1a129a56c29c admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:01.307228 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa39937e-cb8d-4c29-88c5-1a129a56c29c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:01.309137 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa39937e-cb8d-4c29-88c5-1a129a56c29c admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:01.320845 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1266/2512] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:19:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:19:01.326548 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f6d617-b266-4f9b-b0db-7d4531a88511 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:01.337052 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f6d617-b266-4f9b-b0db-7d4531a88511 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:01.342499 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5060ab8-3bb1-4112-aa36-1ec12db57f2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:01.343202 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5060ab8-3bb1-4112-aa36-1ec12db57f2e admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:01.343480 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5060ab8-3bb1-4112-aa36-1ec12db57f2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:01.343753 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5060ab8-3bb1-4112-aa36-1ec12db57f2e admin admin] PATH_INFO: `/v3/projects/a09608c4844d411597d3662a4324d043/users/152946f0fbb04b0e831c7b3ef91140ec/roles/eecd2f160dd844f5906f3d345780c501` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:01.360924 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5060ab8-3bb1-4112-aa36-1ec12db57f2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a09608c4844d411597d3662a4324d043, user_id=152946f0fbb04b0e831c7b3ef91140ec, role_id=eecd2f160dd844f5906f3d345780c501)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:01.363612 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5060ab8-3bb1-4112-aa36-1ec12db57f2e admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:01.381273 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1247/2513] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:19:01 2022] PUT /identity/v3/projects/a09608c4844d411597d3662a4324d043/users/152946f0fbb04b0e831c7b3ef91140ec/roles/eecd2f160dd844f5906f3d345780c501 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:19:01.452139 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac1aed7-ffd8-4fc5-81aa-b0b0fe2a162a None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:01.452421 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac1aed7-ffd8-4fc5-81aa-b0b0fe2a162a None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:01.452712 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac1aed7-ffd8-4fc5-81aa-b0b0fe2a162a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:01.480404 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-aac1aed7-ffd8-4fc5-81aa-b0b0fe2a162a None None] MFA Rules not processed for user `152946f0fbb04b0e831c7b3ef91140ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:19:01.481590 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-aac1aed7-ffd8-4fc5-81aa-b0b0fe2a162a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:19:01.545058 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1267/2514] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:19:01 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:19:01.549456 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5060ab8-3bb1-4112-aa36-1ec12db57f2e admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:01.610319 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5060ab8-3bb1-4112-aa36-1ec12db57f2e admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:01.614807 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d0338a-9143-412f-9cb1-6794f10367f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:01.615422 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d0338a-9143-412f-9cb1-6794f10367f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:01.615696 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d0338a-9143-412f-9cb1-6794f10367f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:01.615962 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d0338a-9143-412f-9cb1-6794f10367f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:01.616571 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d0338a-9143-412f-9cb1-6794f10367f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:01.619484 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d0338a-9143-412f-9cb1-6794f10367f5 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:01.645525 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1248/2515] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 9 12:19:01 2022] POST /identity/v3/projects => generated 365 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:19:01.650362 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac1aed7-ffd8-4fc5-81aa-b0b0fe2a162a None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:01.661052 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac1aed7-ffd8-4fc5-81aa-b0b0fe2a162a None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:01.716298 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5556ac9a-8915-4bdd-877e-e863fd6eff32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:01.716896 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5556ac9a-8915-4bdd-877e-e863fd6eff32 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:01.717175 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5556ac9a-8915-4bdd-877e-e863fd6eff32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:01.717447 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5556ac9a-8915-4bdd-877e-e863fd6eff32 admin admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:01.718067 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5556ac9a-8915-4bdd-877e-e863fd6eff32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:01.720025 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5556ac9a-8915-4bdd-877e-e863fd6eff32 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:01.741817 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1268/2516] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:19:01 2022] POST /identity/v3/users => generated 289 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:19:01.746140 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d0338a-9143-412f-9cb1-6794f10367f5 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:01.757025 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d0338a-9143-412f-9cb1-6794f10367f5 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:01.761746 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de7dda0-b434-46f5-a179-e4479835974e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:01.762363 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de7dda0-b434-46f5-a179-e4479835974e admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:01.762642 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de7dda0-b434-46f5-a179-e4479835974e admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:01.762944 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de7dda0-b434-46f5-a179-e4479835974e admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:01.763652 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de7dda0-b434-46f5-a179-e4479835974e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:01.765249 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de7dda0-b434-46f5-a179-e4479835974e admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:01.776313 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1249/2517] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:19:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:19:01.780395 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5556ac9a-8915-4bdd-877e-e863fd6eff32 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:01.788012 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:19:01.795228 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5556ac9a-8915-4bdd-877e-e863fd6eff32 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:01.805774 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2cde2d-6bf7-450f-8586-dba1aca6856f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:01.806449 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2cde2d-6bf7-450f-8586-dba1aca6856f admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:01.806730 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2cde2d-6bf7-450f-8586-dba1aca6856f admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:01.807028 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2cde2d-6bf7-450f-8586-dba1aca6856f admin admin] PATH_INFO: `/v3/projects/23d574c5b5ed4b239b3dfcf2a3f361dd/users/348db204a7414ab28d61e69fb747ff43/roles/3f2ca33902a04b6dbc2ef7d06d7289d3` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:01.823405 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2cde2d-6bf7-450f-8586-dba1aca6856f admin admin] RBAC: Authorizing `identity:create_grant(project_id=23d574c5b5ed4b239b3dfcf2a3f361dd, user_id=348db204a7414ab28d61e69fb747ff43, role_id=3f2ca33902a04b6dbc2ef7d06d7289d3)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:01.826125 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2cde2d-6bf7-450f-8586-dba1aca6856f admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:01.839984 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1269/2518] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:19:01 2022] PUT /identity/v3/projects/23d574c5b5ed4b239b3dfcf2a3f361dd/users/348db204a7414ab28d61e69fb747ff43/roles/3f2ca33902a04b6dbc2ef7d06d7289d3 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:19:01.844040 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de7dda0-b434-46f5-a179-e4479835974e admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:01.854890 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de7dda0-b434-46f5-a179-e4479835974e admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:01.887040 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:19:01.916341 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a35f7a-fbf9-433d-b96b-33c10c9261d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:01.916960 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a35f7a-fbf9-433d-b96b-33c10c9261d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:01.917239 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a35f7a-fbf9-433d-b96b-33c10c9261d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:01.917511 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a35f7a-fbf9-433d-b96b-33c10c9261d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:01.918163 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a35f7a-fbf9-433d-b96b-33c10c9261d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:01.920886 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a35f7a-fbf9-433d-b96b-33c10c9261d5 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:01.934948 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1250/2519] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:19:01 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 12:19:01.936793 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2cde2d-6bf7-450f-8586-dba1aca6856f admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:01.949786 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2cde2d-6bf7-450f-8586-dba1aca6856f admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:01.954560 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5188e8b8-414c-4131-b829-32bd3136e8a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:01.955253 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5188e8b8-414c-4131-b829-32bd3136e8a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:01.955539 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5188e8b8-414c-4131-b829-32bd3136e8a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:01.955810 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5188e8b8-414c-4131-b829-32bd3136e8a0 admin admin] PATH_INFO: `/v3/projects/23d574c5b5ed4b239b3dfcf2a3f361dd/users/348db204a7414ab28d61e69fb747ff43/roles/3f2ca33902a04b6dbc2ef7d06d7289d3` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:01.957881 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5188e8b8-414c-4131-b829-32bd3136e8a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23d574c5b5ed4b239b3dfcf2a3f361dd, user_id=348db204a7414ab28d61e69fb747ff43, role_id=3f2ca33902a04b6dbc2ef7d06d7289d3)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:01.960912 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5188e8b8-414c-4131-b829-32bd3136e8a0 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:01.973752 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1270/2520] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:19:01 2022] PUT /identity/v3/projects/23d574c5b5ed4b239b3dfcf2a3f361dd/users/348db204a7414ab28d61e69fb747ff43/roles/3f2ca33902a04b6dbc2ef7d06d7289d3 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:19:01.984758 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b8ad9a-d85e-401b-b230-24f10f6201a9 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:01.985050 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b8ad9a-d85e-401b-b230-24f10f6201a9 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:01.985328 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b8ad9a-d85e-401b-b230-24f10f6201a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:02.017760 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-99b8ad9a-d85e-401b-b230-24f10f6201a9 None None] MFA Rules not processed for user `348db204a7414ab28d61e69fb747ff43`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:19:02.019021 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-99b8ad9a-d85e-401b-b230-24f10f6201a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:19:02.099654 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1251/2521] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:19:01 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:19:02.111621 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5188e8b8-414c-4131-b829-32bd3136e8a0 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:02.113302 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-5188e8b8-414c-4131-b829-32bd3136e8a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:19:02.171045 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5188e8b8-414c-4131-b829-32bd3136e8a0 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:02.174682 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667177c1-fe97-4bb4-88dc-7ccf41dd4905 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535c258100df40b58a5ee44b2c21d621', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:02.175316 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667177c1-fe97-4bb4-88dc-7ccf41dd4905 service cinder] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:02.175595 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667177c1-fe97-4bb4-88dc-7ccf41dd4905 service cinder] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:02.175868 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667177c1-fe97-4bb4-88dc-7ccf41dd4905 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:02.185725 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667177c1-fe97-4bb4-88dc-7ccf41dd4905 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:02.187172 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667177c1-fe97-4bb4-88dc-7ccf41dd4905 service cinder] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:02.198023 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1271/2522] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:19:02 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:19:02.285357 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b8ad9a-d85e-401b-b230-24f10f6201a9 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:02.295345 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b8ad9a-d85e-401b-b230-24f10f6201a9 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:02.299171 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb545e5-dd72-4df3-98c0-edaa5adbe222 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535c258100df40b58a5ee44b2c21d621', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:02.299745 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb545e5-dd72-4df3-98c0-edaa5adbe222 service cinder] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:02.300024 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb545e5-dd72-4df3-98c0-edaa5adbe222 service cinder] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:02.300297 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb545e5-dd72-4df3-98c0-edaa5adbe222 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:02.358645 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb545e5-dd72-4df3-98c0-edaa5adbe222 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:02.360207 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb545e5-dd72-4df3-98c0-edaa5adbe222 service cinder] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:02.369782 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1252/2523] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:19:02 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:19:07.140150 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667177c1-fe97-4bb4-88dc-7ccf41dd4905 service cinder] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:07.207077 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667177c1-fe97-4bb4-88dc-7ccf41dd4905 service cinder] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:07.212561 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12ec7ff-7a9a-41c1-8e35-851fb380b65a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:07.213209 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12ec7ff-7a9a-41c1-8e35-851fb380b65a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:07.213487 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12ec7ff-7a9a-41c1-8e35-851fb380b65a admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:07.213754 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12ec7ff-7a9a-41c1-8e35-851fb380b65a admin admin] PATH_INFO: `/v3/users/152946f0fbb04b0e831c7b3ef91140ec` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:07.214888 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12ec7ff-7a9a-41c1-8e35-851fb380b65a admin admin] RBAC: Authorizing `identity:delete_user(user_id=152946f0fbb04b0e831c7b3ef91140ec)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:07.216882 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12ec7ff-7a9a-41c1-8e35-851fb380b65a admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:07.266073 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-d12ec7ff-7a9a-41c1-8e35-851fb380b65a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '152946f0fbb04b0e831c7b3ef91140ec'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:07.273047 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-d12ec7ff-7a9a-41c1-8e35-851fb380b65a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '152946f0fbb04b0e831c7b3ef91140ec'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:07.284284 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:19:07.286167 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-d12ec7ff-7a9a-41c1-8e35-851fb380b65a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '152946f0fbb04b0e831c7b3ef91140ec'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:07.294282 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-d12ec7ff-7a9a-41c1-8e35-851fb380b65a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '152946f0fbb04b0e831c7b3ef91140ec'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:07.295172 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-d12ec7ff-7a9a-41c1-8e35-851fb380b65a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '152946f0fbb04b0e831c7b3ef91140ec'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:07.301367 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1272/2524] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 9 12:19:07 2022] DELETE /identity/v3/users/152946f0fbb04b0e831c7b3ef91140ec => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:19:07.306524 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64da4cf3-1b3f-474b-a43b-f421aa766c7f None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:07.307023 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64da4cf3-1b3f-474b-a43b-f421aa766c7f None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:07.307123 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64da4cf3-1b3f-474b-a43b-f421aa766c7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:07.326891 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-64da4cf3-1b3f-474b-a43b-f421aa766c7f None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:19:07.328333 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-64da4cf3-1b3f-474b-a43b-f421aa766c7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:19:07.376043 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:19:07.387961 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1253/2525] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:19:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:19:07.397932 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12ec7ff-7a9a-41c1-8e35-851fb380b65a admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:07.400092 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-d12ec7ff-7a9a-41c1-8e35-851fb380b65a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:19:07.458817 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12ec7ff-7a9a-41c1-8e35-851fb380b65a admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:07.463186 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6409ae1-c46a-422f-b3e5-20f58f770e96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:07.463797 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6409ae1-c46a-422f-b3e5-20f58f770e96 service neutron] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:07.464083 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6409ae1-c46a-422f-b3e5-20f58f770e96 service neutron] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:07.464357 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6409ae1-c46a-422f-b3e5-20f58f770e96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:07.474487 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6409ae1-c46a-422f-b3e5-20f58f770e96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:07.475962 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6409ae1-c46a-422f-b3e5-20f58f770e96 service neutron] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:07.486016 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1273/2526] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:19:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:19:07.656531 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ec95d4-41ff-48b5-8677-142aa1822d8c None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:07.656932 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ec95d4-41ff-48b5-8677-142aa1822d8c None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:07.657280 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ec95d4-41ff-48b5-8677-142aa1822d8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:07.680392 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-c8ec95d4-41ff-48b5-8677-142aa1822d8c None None] MFA Rules not processed for user `e368b4e668c54e25bff80a8e8a8c3588`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:19:07.681737 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-c8ec95d4-41ff-48b5-8677-142aa1822d8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:19:07.743255 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1254/2527] 10.222.0.49 () {58 vars in 1009 bytes} [Fri Sep 9 12:19:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:19:07.758602 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6409ae1-c46a-422f-b3e5-20f58f770e96 service neutron] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:07.760329 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-c6409ae1-c46a-422f-b3e5-20f58f770e96 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:19:07.775048 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ec95d4-41ff-48b5-8677-142aa1822d8c None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:07.776439 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-c8ec95d4-41ff-48b5-8677-142aa1822d8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:19:07.792103 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ec95d4-41ff-48b5-8677-142aa1822d8c None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:07.799727 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148be971-084f-4e5f-98d4-806acc6c326c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:07.800730 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148be971-084f-4e5f-98d4-806acc6c326c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:07.801011 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148be971-084f-4e5f-98d4-806acc6c326c admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:07.801278 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148be971-084f-4e5f-98d4-806acc6c326c admin admin] PATH_INFO: `/v3/projects/a09608c4844d411597d3662a4324d043` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:07.802553 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148be971-084f-4e5f-98d4-806acc6c326c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a09608c4844d411597d3662a4324d043)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:07.805266 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148be971-084f-4e5f-98d4-806acc6c326c admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:07.846984 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6409ae1-c46a-422f-b3e5-20f58f770e96 service neutron] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:07.852655 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4a0dc3-1099-4bf3-ba30-63142e55d89a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e652ba827849b5a3fc1213d04a1b76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:07.853274 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4a0dc3-1099-4bf3-ba30-63142e55d89a service swift] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:07.853581 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4a0dc3-1099-4bf3-ba30-63142e55d89a service swift] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:07.854042 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4a0dc3-1099-4bf3-ba30-63142e55d89a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:07.863261 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4a0dc3-1099-4bf3-ba30-63142e55d89a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:07.864903 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4a0dc3-1099-4bf3-ba30-63142e55d89a service swift] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:07.869431 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-148be971-084f-4e5f-98d4-806acc6c326c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a09608c4844d411597d3662a4324d043', 'request_id': 'req-148be971-084f-4e5f-98d4-806acc6c326c'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:07.875915 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1274/2528] 10.222.0.49 () {60 vars in 1384 bytes} [Fri Sep 9 12:19:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 12:19:07.876270 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-148be971-084f-4e5f-98d4-806acc6c326c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a09608c4844d411597d3662a4324d043', 'request_id': 'req-148be971-084f-4e5f-98d4-806acc6c326c'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:07.884544 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-148be971-084f-4e5f-98d4-806acc6c326c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a09608c4844d411597d3662a4324d043', 'request_id': 'req-148be971-084f-4e5f-98d4-806acc6c326c'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:07.899459 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-148be971-084f-4e5f-98d4-806acc6c326c admin admin] The token cache is being invalidate because project a09608c4844d411597d3662a4324d043 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:19:07.900071 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-148be971-084f-4e5f-98d4-806acc6c326c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:07.900883 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-148be971-084f-4e5f-98d4-806acc6c326c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:07.902668 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1255/2529] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 9 12:19:07 2022] DELETE /identity/v3/projects/a09608c4844d411597d3662a4324d043 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:19:07.906684 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4a0dc3-1099-4bf3-ba30-63142e55d89a service swift] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:07.918670 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4a0dc3-1099-4bf3-ba30-63142e55d89a service swift] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:07.988496 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7c9528-865b-4c16-90b7-0d8e78e27673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:07.989141 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7c9528-865b-4c16-90b7-0d8e78e27673 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:07.989423 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7c9528-865b-4c16-90b7-0d8e78e27673 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:07.989695 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7c9528-865b-4c16-90b7-0d8e78e27673 admin admin] PATH_INFO: `/v3/users/348db204a7414ab28d61e69fb747ff43` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:07.990856 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7c9528-865b-4c16-90b7-0d8e78e27673 admin admin] RBAC: Authorizing `identity:delete_user(user_id=348db204a7414ab28d61e69fb747ff43)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:07.993746 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7c9528-865b-4c16-90b7-0d8e78e27673 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:08.034932 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:19:08.073789 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-1d7c9528-865b-4c16-90b7-0d8e78e27673 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '348db204a7414ab28d61e69fb747ff43'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:08.090463 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-1d7c9528-865b-4c16-90b7-0d8e78e27673 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '348db204a7414ab28d61e69fb747ff43'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:08.111041 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-1d7c9528-865b-4c16-90b7-0d8e78e27673 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '348db204a7414ab28d61e69fb747ff43'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:08.122249 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-1d7c9528-865b-4c16-90b7-0d8e78e27673 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '348db204a7414ab28d61e69fb747ff43'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:08.123239 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-1d7c9528-865b-4c16-90b7-0d8e78e27673 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '348db204a7414ab28d61e69fb747ff43'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:08.136769 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1275/2530] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 9 12:19:07 2022] DELETE /identity/v3/users/348db204a7414ab28d61e69fb747ff43 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:19:08.507110 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148be971-084f-4e5f-98d4-806acc6c326c admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:08.509255 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-148be971-084f-4e5f-98d4-806acc6c326c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:19:08.546991 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148be971-084f-4e5f-98d4-806acc6c326c admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:08.683198 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab5c117-e205-4c9c-bf55-2e224f4d35dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:08.683809 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab5c117-e205-4c9c-bf55-2e224f4d35dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:08.684303 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab5c117-e205-4c9c-bf55-2e224f4d35dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:08.684456 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab5c117-e205-4c9c-bf55-2e224f4d35dc admin admin] PATH_INFO: `/v3/projects/23d574c5b5ed4b239b3dfcf2a3f361dd` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:08.685616 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab5c117-e205-4c9c-bf55-2e224f4d35dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=23d574c5b5ed4b239b3dfcf2a3f361dd)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:08.688094 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab5c117-e205-4c9c-bf55-2e224f4d35dc admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:08.735166 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:19:08.772381 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-eab5c117-e205-4c9c-bf55-2e224f4d35dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23d574c5b5ed4b239b3dfcf2a3f361dd', 'request_id': 'req-eab5c117-e205-4c9c-bf55-2e224f4d35dc'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:08.779335 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-eab5c117-e205-4c9c-bf55-2e224f4d35dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23d574c5b5ed4b239b3dfcf2a3f361dd', 'request_id': 'req-eab5c117-e205-4c9c-bf55-2e224f4d35dc'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:08.788474 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-eab5c117-e205-4c9c-bf55-2e224f4d35dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23d574c5b5ed4b239b3dfcf2a3f361dd', 'request_id': 'req-eab5c117-e205-4c9c-bf55-2e224f4d35dc'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:08.802104 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-eab5c117-e205-4c9c-bf55-2e224f4d35dc admin admin] The token cache is being invalidate because project 23d574c5b5ed4b239b3dfcf2a3f361dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:19:08.802464 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-eab5c117-e205-4c9c-bf55-2e224f4d35dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:08.803490 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-eab5c117-e205-4c9c-bf55-2e224f4d35dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:08.805971 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1256/2531] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 9 12:19:08 2022] DELETE /identity/v3/projects/23d574c5b5ed4b239b3dfcf2a3f361dd => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:19:08.811388 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c1bbfa-a9d6-4d9b-a4d4-d97fef978b3b None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:08.813667 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c1bbfa-a9d6-4d9b-a4d4-d97fef978b3b None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:08.814308 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c1bbfa-a9d6-4d9b-a4d4-d97fef978b3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:08.835372 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-49c1bbfa-a9d6-4d9b-a4d4-d97fef978b3b None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:19:08.836728 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-49c1bbfa-a9d6-4d9b-a4d4-d97fef978b3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:19:08.907446 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1276/2532] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:19:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:19:08.917063 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1158462-f7b9-4e60-b469-e7ff1a55cf15 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:08.917377 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1158462-f7b9-4e60-b469-e7ff1a55cf15 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:08.917636 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1158462-f7b9-4e60-b469-e7ff1a55cf15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:08.940071 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-a1158462-f7b9-4e60-b469-e7ff1a55cf15 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:19:08.941662 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-a1158462-f7b9-4e60-b469-e7ff1a55cf15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:19:08.956308 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1257/2533] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:19:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:19:08.957482 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c1bbfa-a9d6-4d9b-a4d4-d97fef978b3b None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:08.969920 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c1bbfa-a9d6-4d9b-a4d4-d97fef978b3b None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:08.983645 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8abfdc-cf56-4727-a545-e0ed11061e7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:08.984573 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8abfdc-cf56-4727-a545-e0ed11061e7c admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:08.984693 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8abfdc-cf56-4727-a545-e0ed11061e7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:08.985042 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8abfdc-cf56-4727-a545-e0ed11061e7c admin admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:08.985615 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8abfdc-cf56-4727-a545-e0ed11061e7c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:19:08.985927 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8abfdc-cf56-4727-a545-e0ed11061e7c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:08.987631 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8abfdc-cf56-4727-a545-e0ed11061e7c admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:09.004505 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1277/2534] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 9 12:19:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:19:09.008659 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1158462-f7b9-4e60-b469-e7ff1a55cf15 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:09.022884 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1158462-f7b9-4e60-b469-e7ff1a55cf15 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:09.028101 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddec221-5e22-476a-af69-462e66d88a40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:09.029012 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddec221-5e22-476a-af69-462e66d88a40 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:09.029297 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddec221-5e22-476a-af69-462e66d88a40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:09.029584 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddec221-5e22-476a-af69-462e66d88a40 admin admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:09.040473 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddec221-5e22-476a-af69-462e66d88a40 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:09.042669 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddec221-5e22-476a-af69-462e66d88a40 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:09.075824 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1258/2535] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 9 12:19:09 2022] POST /identity/v3/projects => generated 371 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:19:09.079922 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8abfdc-cf56-4727-a545-e0ed11061e7c admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:09.090533 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8abfdc-cf56-4727-a545-e0ed11061e7c admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:09.164677 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836e10a2-dc3c-4ac9-a55e-c10277425f39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:09.165784 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836e10a2-dc3c-4ac9-a55e-c10277425f39 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:09.166105 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836e10a2-dc3c-4ac9-a55e-c10277425f39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:09.166424 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836e10a2-dc3c-4ac9-a55e-c10277425f39 admin admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:09.167190 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836e10a2-dc3c-4ac9-a55e-c10277425f39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:09.169632 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836e10a2-dc3c-4ac9-a55e-c10277425f39 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:09.196595 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1278/2536] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:19:09 2022] POST /identity/v3/users => generated 286 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:19:09.197989 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddec221-5e22-476a-af69-462e66d88a40 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:09.216079 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddec221-5e22-476a-af69-462e66d88a40 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:09.220946 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dfb4e9-489b-46a2-a32c-43d763e9d0a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:09.221568 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dfb4e9-489b-46a2-a32c-43d763e9d0a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:09.221844 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dfb4e9-489b-46a2-a32c-43d763e9d0a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:09.222121 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dfb4e9-489b-46a2-a32c-43d763e9d0a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:09.222794 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dfb4e9-489b-46a2-a32c-43d763e9d0a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:09.224454 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dfb4e9-489b-46a2-a32c-43d763e9d0a8 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:09.240297 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1259/2537] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:19:09 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 12:19:09.242082 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836e10a2-dc3c-4ac9-a55e-c10277425f39 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:09.252886 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836e10a2-dc3c-4ac9-a55e-c10277425f39 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:09.258168 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2634219a-c5da-46d4-8d93-03ef108104a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:09.258913 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2634219a-c5da-46d4-8d93-03ef108104a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:09.259346 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2634219a-c5da-46d4-8d93-03ef108104a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:09.259650 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2634219a-c5da-46d4-8d93-03ef108104a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:09.260293 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2634219a-c5da-46d4-8d93-03ef108104a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:09.261968 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2634219a-c5da-46d4-8d93-03ef108104a5 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:09.275313 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1279/2538] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:19:09 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:19:09.279408 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dfb4e9-489b-46a2-a32c-43d763e9d0a8 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:09.291007 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dfb4e9-489b-46a2-a32c-43d763e9d0a8 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:09.295817 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8bcc86-b5c9-4eb4-a19b-74c7211829bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:09.296800 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8bcc86-b5c9-4eb4-a19b-74c7211829bd admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:09.296994 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8bcc86-b5c9-4eb4-a19b-74c7211829bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:09.297275 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8bcc86-b5c9-4eb4-a19b-74c7211829bd admin admin] PATH_INFO: `/v3/projects/3157db5219744ceaa0bad0de296bdced/users/fd92ffe5e6fb45cbbdfeb1df1f64076e/roles/eecd2f160dd844f5906f3d345780c501` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:09.314950 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8bcc86-b5c9-4eb4-a19b-74c7211829bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=3157db5219744ceaa0bad0de296bdced, user_id=fd92ffe5e6fb45cbbdfeb1df1f64076e, role_id=eecd2f160dd844f5906f3d345780c501)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:09.317714 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8bcc86-b5c9-4eb4-a19b-74c7211829bd admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:09.331705 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1260/2539] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:19:09 2022] PUT /identity/v3/projects/3157db5219744ceaa0bad0de296bdced/users/fd92ffe5e6fb45cbbdfeb1df1f64076e/roles/eecd2f160dd844f5906f3d345780c501 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:19:09.337585 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6308bd41-2128-4e59-b2be-09ea2c308610 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:09.337869 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6308bd41-2128-4e59-b2be-09ea2c308610 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:09.338143 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6308bd41-2128-4e59-b2be-09ea2c308610 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:09.346945 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:19:09.361745 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-6308bd41-2128-4e59-b2be-09ea2c308610 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:19:09.363367 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-6308bd41-2128-4e59-b2be-09ea2c308610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:19:09.442653 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1280/2540] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:19:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:19:09.450069 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8bcc86-b5c9-4eb4-a19b-74c7211829bd admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:09.451854 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-7a8bcc86-b5c9-4eb4-a19b-74c7211829bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:19:09.519025 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8bcc86-b5c9-4eb4-a19b-74c7211829bd admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:09.523812 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd90f772-ee5f-4e2f-8cc4-f118a69555ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:09.524567 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd90f772-ee5f-4e2f-8cc4-f118a69555ea service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:09.524916 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd90f772-ee5f-4e2f-8cc4-f118a69555ea service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:09.525265 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd90f772-ee5f-4e2f-8cc4-f118a69555ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:09.536216 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd90f772-ee5f-4e2f-8cc4-f118a69555ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:09.537878 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd90f772-ee5f-4e2f-8cc4-f118a69555ea service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:09.549101 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1261/2541] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:19:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:19:10.363608 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5dc7be-d08c-4aa5-b75d-c06841758f7d None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:10.364113 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5dc7be-d08c-4aa5-b75d-c06841758f7d None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:10.364213 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5dc7be-d08c-4aa5-b75d-c06841758f7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:10.409937 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-9d5dc7be-d08c-4aa5-b75d-c06841758f7d None None] MFA Rules not processed for user `fd92ffe5e6fb45cbbdfeb1df1f64076e`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:19:10.411281 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-9d5dc7be-d08c-4aa5-b75d-c06841758f7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:19:10.540256 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1281/2542] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:19:10 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:19:10.550452 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd90f772-ee5f-4e2f-8cc4-f118a69555ea service neutron] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:10.562564 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd90f772-ee5f-4e2f-8cc4-f118a69555ea service neutron] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:10.568004 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee8f924-04d5-425a-92ee-f8e2af5cb391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:10.568715 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee8f924-04d5-425a-92ee-f8e2af5cb391 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:10.569080 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee8f924-04d5-425a-92ee-f8e2af5cb391 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:10.569594 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee8f924-04d5-425a-92ee-f8e2af5cb391 admin admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:10.571154 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee8f924-04d5-425a-92ee-f8e2af5cb391 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:10.573271 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee8f924-04d5-425a-92ee-f8e2af5cb391 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:10.621444 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1262/2543] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 9 12:19:10 2022] POST /identity/v3/projects => generated 371 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:19:10.625375 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5dc7be-d08c-4aa5-b75d-c06841758f7d None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:10.639450 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5dc7be-d08c-4aa5-b75d-c06841758f7d None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:10.718130 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09ece4c-688b-4f89-982a-fd7c1e5974fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:10.718863 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09ece4c-688b-4f89-982a-fd7c1e5974fe admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:10.719177 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09ece4c-688b-4f89-982a-fd7c1e5974fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:10.719453 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09ece4c-688b-4f89-982a-fd7c1e5974fe admin admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:10.720157 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09ece4c-688b-4f89-982a-fd7c1e5974fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:10.722098 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09ece4c-688b-4f89-982a-fd7c1e5974fe admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:10.758931 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1282/2544] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:19:10 2022] POST /identity/v3/users => generated 292 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:19:10.760331 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee8f924-04d5-425a-92ee-f8e2af5cb391 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:10.772852 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee8f924-04d5-425a-92ee-f8e2af5cb391 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:10.787793 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee13af9c-8d95-4bbf-bc0a-f88b098ca25c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:10.788613 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee13af9c-8d95-4bbf-bc0a-f88b098ca25c admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:10.789030 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee13af9c-8d95-4bbf-bc0a-f88b098ca25c admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:10.789601 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee13af9c-8d95-4bbf-bc0a-f88b098ca25c admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:10.790682 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee13af9c-8d95-4bbf-bc0a-f88b098ca25c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:10.792419 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee13af9c-8d95-4bbf-bc0a-f88b098ca25c admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:10.806216 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1263/2545] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:19:10 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:19:10.810419 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09ece4c-688b-4f89-982a-fd7c1e5974fe admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:10.825864 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09ece4c-688b-4f89-982a-fd7c1e5974fe admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:10.831842 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91d8ad7-72dd-4e5e-8616-d294bf06f986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:10.832731 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91d8ad7-72dd-4e5e-8616-d294bf06f986 admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:10.833222 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91d8ad7-72dd-4e5e-8616-d294bf06f986 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:10.835915 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91d8ad7-72dd-4e5e-8616-d294bf06f986 admin admin] PATH_INFO: `/v3/projects/07331daf772d474b860f5f0ac0a5757d/users/15350e66e91e4384b89eab95ca51ee03/roles/3f2ca33902a04b6dbc2ef7d06d7289d3` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:10.852407 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91d8ad7-72dd-4e5e-8616-d294bf06f986 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07331daf772d474b860f5f0ac0a5757d, user_id=15350e66e91e4384b89eab95ca51ee03, role_id=3f2ca33902a04b6dbc2ef7d06d7289d3)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:10.855174 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91d8ad7-72dd-4e5e-8616-d294bf06f986 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:10.864463 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:19:10.873327 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1283/2546] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:19:10 2022] PUT /identity/v3/projects/07331daf772d474b860f5f0ac0a5757d/users/15350e66e91e4384b89eab95ca51ee03/roles/3f2ca33902a04b6dbc2ef7d06d7289d3 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:19:10.875249 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee13af9c-8d95-4bbf-bc0a-f88b098ca25c admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:10.887619 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee13af9c-8d95-4bbf-bc0a-f88b098ca25c admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:10.906303 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:19:10.954853 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f018bb4-a04c-4fc7-84e3-bf3773654030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:10.955496 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f018bb4-a04c-4fc7-84e3-bf3773654030 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:10.955930 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f018bb4-a04c-4fc7-84e3-bf3773654030 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:10.956206 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f018bb4-a04c-4fc7-84e3-bf3773654030 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:10.956843 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f018bb4-a04c-4fc7-84e3-bf3773654030 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:10.958669 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f018bb4-a04c-4fc7-84e3-bf3773654030 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:10.973617 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1264/2547] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:19:10 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:19:10.974848 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91d8ad7-72dd-4e5e-8616-d294bf06f986 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:10.989889 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91d8ad7-72dd-4e5e-8616-d294bf06f986 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:10.999458 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc98bacc-e19a-4615-9acb-a0d7446be388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:11.000140 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc98bacc-e19a-4615-9acb-a0d7446be388 admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:11.000433 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc98bacc-e19a-4615-9acb-a0d7446be388 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:11.000705 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc98bacc-e19a-4615-9acb-a0d7446be388 admin admin] PATH_INFO: `/v3/projects/07331daf772d474b860f5f0ac0a5757d/users/15350e66e91e4384b89eab95ca51ee03/roles/3f2ca33902a04b6dbc2ef7d06d7289d3` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:11.002633 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc98bacc-e19a-4615-9acb-a0d7446be388 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07331daf772d474b860f5f0ac0a5757d, user_id=15350e66e91e4384b89eab95ca51ee03, role_id=3f2ca33902a04b6dbc2ef7d06d7289d3)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:11.005288 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc98bacc-e19a-4615-9acb-a0d7446be388 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:11.017708 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1284/2548] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:19:10 2022] PUT /identity/v3/projects/07331daf772d474b860f5f0ac0a5757d/users/15350e66e91e4384b89eab95ca51ee03/roles/3f2ca33902a04b6dbc2ef7d06d7289d3 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:19:11.891390 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2852a7-7583-433f-afce-db54c634fcb2 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:11.897080 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2852a7-7583-433f-afce-db54c634fcb2 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:11.897080 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2852a7-7583-433f-afce-db54c634fcb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:11.938429 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-de2852a7-7583-433f-afce-db54c634fcb2 None None] MFA Rules not processed for user `15350e66e91e4384b89eab95ca51ee03`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:19:11.939482 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-de2852a7-7583-433f-afce-db54c634fcb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:19:12.032239 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1265/2549] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:19:11 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:19:12.044839 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc98bacc-e19a-4615-9acb-a0d7446be388 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:12.046892 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-cc98bacc-e19a-4615-9acb-a0d7446be388 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:19:12.147693 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc98bacc-e19a-4615-9acb-a0d7446be388 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:12.153396 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8cbeba-878c-49a8-9417-731de2e1626c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535c258100df40b58a5ee44b2c21d621', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:12.154537 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8cbeba-878c-49a8-9417-731de2e1626c service cinder] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:12.155044 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8cbeba-878c-49a8-9417-731de2e1626c service cinder] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:12.155387 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8cbeba-878c-49a8-9417-731de2e1626c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:12.242967 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8cbeba-878c-49a8-9417-731de2e1626c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:12.245366 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8cbeba-878c-49a8-9417-731de2e1626c service cinder] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:12.256566 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1285/2550] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:19:12 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:19:13.685996 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2852a7-7583-433f-afce-db54c634fcb2 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:13.687976 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-de2852a7-7583-433f-afce-db54c634fcb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:19:13.705818 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2852a7-7583-433f-afce-db54c634fcb2 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:13.712194 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1985617b-dd34-4f44-8f66-ce227522e74d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535c258100df40b58a5ee44b2c21d621', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:13.712837 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1985617b-dd34-4f44-8f66-ce227522e74d service cinder] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:13.713187 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1985617b-dd34-4f44-8f66-ce227522e74d service cinder] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:13.713576 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1985617b-dd34-4f44-8f66-ce227522e74d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:13.723634 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1985617b-dd34-4f44-8f66-ce227522e74d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:13.725300 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1985617b-dd34-4f44-8f66-ce227522e74d service cinder] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:13.737611 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1266/2551] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:19:13 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:19:22.084024 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d217ff57-b7ca-4e9f-b13d-0adfa9962ce8 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:22.084622 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d217ff57-b7ca-4e9f-b13d-0adfa9962ce8 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:22.084622 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d217ff57-b7ca-4e9f-b13d-0adfa9962ce8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:22.105758 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-d217ff57-b7ca-4e9f-b13d-0adfa9962ce8 None None] MFA Rules not processed for user `e368b4e668c54e25bff80a8e8a8c3588`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:19:22.107017 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-d217ff57-b7ca-4e9f-b13d-0adfa9962ce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:19:22.159282 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1286/2552] 10.222.0.49 () {58 vars in 967 bytes} [Fri Sep 9 12:19:22 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:19:22.172221 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1985617b-dd34-4f44-8f66-ce227522e74d service cinder] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:22.173663 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-1985617b-dd34-4f44-8f66-ce227522e74d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:19:22.243990 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1985617b-dd34-4f44-8f66-ce227522e74d service cinder] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:22.248933 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d765abd7-9ce8-4403-b4b9-28fded65d3c1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e652ba827849b5a3fc1213d04a1b76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:22.249531 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d765abd7-9ce8-4403-b4b9-28fded65d3c1 service swift] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:22.249806 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d765abd7-9ce8-4403-b4b9-28fded65d3c1 service swift] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:22.250075 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d765abd7-9ce8-4403-b4b9-28fded65d3c1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:22.258530 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d765abd7-9ce8-4403-b4b9-28fded65d3c1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:22.260149 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d765abd7-9ce8-4403-b4b9-28fded65d3c1 service swift] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:22.268299 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1267/2553] 10.222.0.49 () {60 vars in 1384 bytes} [Fri Sep 9 12:19:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 12:19:26.954056 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d217ff57-b7ca-4e9f-b13d-0adfa9962ce8 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:27.003012 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:19:27.019858 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d217ff57-b7ca-4e9f-b13d-0adfa9962ce8 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:27.024930 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4166d39-b980-49ef-9071-27a589f7c648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:27.025634 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4166d39-b980-49ef-9071-27a589f7c648 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:27.025913 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4166d39-b980-49ef-9071-27a589f7c648 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:27.026190 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4166d39-b980-49ef-9071-27a589f7c648 admin admin] PATH_INFO: `/v3/users/fd92ffe5e6fb45cbbdfeb1df1f64076e` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:27.027342 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4166d39-b980-49ef-9071-27a589f7c648 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd92ffe5e6fb45cbbdfeb1df1f64076e)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:27.029245 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4166d39-b980-49ef-9071-27a589f7c648 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:27.080014 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-e4166d39-b980-49ef-9071-27a589f7c648 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd92ffe5e6fb45cbbdfeb1df1f64076e'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:27.088201 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-e4166d39-b980-49ef-9071-27a589f7c648 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd92ffe5e6fb45cbbdfeb1df1f64076e'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:27.101063 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-e4166d39-b980-49ef-9071-27a589f7c648 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd92ffe5e6fb45cbbdfeb1df1f64076e'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:27.108449 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-e4166d39-b980-49ef-9071-27a589f7c648 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd92ffe5e6fb45cbbdfeb1df1f64076e'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:27.109094 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-e4166d39-b980-49ef-9071-27a589f7c648 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd92ffe5e6fb45cbbdfeb1df1f64076e'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:27.114958 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1287/2554] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 9 12:19:26 2022] DELETE /identity/v3/users/fd92ffe5e6fb45cbbdfeb1df1f64076e => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:19:27.307007 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d765abd7-9ce8-4403-b4b9-28fded65d3c1 service swift] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:27.319345 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d765abd7-9ce8-4403-b4b9-28fded65d3c1 service swift] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:27.341121 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=100 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:19:27.387110 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bbe766-6587-45e1-b967-c5268ccc9040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:27.387750 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bbe766-6587-45e1-b967-c5268ccc9040 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:27.388030 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bbe766-6587-45e1-b967-c5268ccc9040 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:27.388309 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bbe766-6587-45e1-b967-c5268ccc9040 admin admin] PATH_INFO: `/v3/projects/3157db5219744ceaa0bad0de296bdced` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:27.389473 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bbe766-6587-45e1-b967-c5268ccc9040 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3157db5219744ceaa0bad0de296bdced)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:27.391860 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bbe766-6587-45e1-b967-c5268ccc9040 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:27.440062 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-a6bbe766-6587-45e1-b967-c5268ccc9040 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3157db5219744ceaa0bad0de296bdced', 'request_id': 'req-a6bbe766-6587-45e1-b967-c5268ccc9040'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:27.446082 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-a6bbe766-6587-45e1-b967-c5268ccc9040 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3157db5219744ceaa0bad0de296bdced', 'request_id': 'req-a6bbe766-6587-45e1-b967-c5268ccc9040'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:27.453244 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-a6bbe766-6587-45e1-b967-c5268ccc9040 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3157db5219744ceaa0bad0de296bdced', 'request_id': 'req-a6bbe766-6587-45e1-b967-c5268ccc9040'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:27.465603 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-a6bbe766-6587-45e1-b967-c5268ccc9040 admin admin] The token cache is being invalidate because project 3157db5219744ceaa0bad0de296bdced was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:19:27.465888 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-a6bbe766-6587-45e1-b967-c5268ccc9040 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:27.466519 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-a6bbe766-6587-45e1-b967-c5268ccc9040 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:27.468142 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1268/2555] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 9 12:19:27 2022] DELETE /identity/v3/projects/3157db5219744ceaa0bad0de296bdced => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:19:27.472073 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4166d39-b980-49ef-9071-27a589f7c648 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:27.473558 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-e4166d39-b980-49ef-9071-27a589f7c648 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:19:27.486637 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4166d39-b980-49ef-9071-27a589f7c648 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:27.552881 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf12a2e-6ccc-476b-b3c1-7ded1efda553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:27.554439 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf12a2e-6ccc-476b-b3c1-7ded1efda553 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:27.554815 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf12a2e-6ccc-476b-b3c1-7ded1efda553 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:27.555239 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf12a2e-6ccc-476b-b3c1-7ded1efda553 admin admin] PATH_INFO: `/v3/users/15350e66e91e4384b89eab95ca51ee03` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:27.556541 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf12a2e-6ccc-476b-b3c1-7ded1efda553 admin admin] RBAC: Authorizing `identity:delete_user(user_id=15350e66e91e4384b89eab95ca51ee03)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:27.559570 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf12a2e-6ccc-476b-b3c1-7ded1efda553 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:27.623288 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-bbf12a2e-6ccc-476b-b3c1-7ded1efda553 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15350e66e91e4384b89eab95ca51ee03'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:27.646469 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-bbf12a2e-6ccc-476b-b3c1-7ded1efda553 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15350e66e91e4384b89eab95ca51ee03'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:27.656657 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:19:27.664130 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-bbf12a2e-6ccc-476b-b3c1-7ded1efda553 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15350e66e91e4384b89eab95ca51ee03'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:27.675466 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-bbf12a2e-6ccc-476b-b3c1-7ded1efda553 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15350e66e91e4384b89eab95ca51ee03'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:27.676990 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-bbf12a2e-6ccc-476b-b3c1-7ded1efda553 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15350e66e91e4384b89eab95ca51ee03'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:27.689650 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1288/2556] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 9 12:19:27 2022] DELETE /identity/v3/users/15350e66e91e4384b89eab95ca51ee03 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:19:27.955974 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bbe766-6587-45e1-b967-c5268ccc9040 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:27.957482 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-a6bbe766-6587-45e1-b967-c5268ccc9040 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:19:27.971199 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bbe766-6587-45e1-b967-c5268ccc9040 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:28.031902 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8dd953-48a3-4d68-acc7-9b95ba878188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:28.032515 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8dd953-48a3-4d68-acc7-9b95ba878188 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:28.032796 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8dd953-48a3-4d68-acc7-9b95ba878188 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:28.033065 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8dd953-48a3-4d68-acc7-9b95ba878188 admin admin] PATH_INFO: `/v3/projects/07331daf772d474b860f5f0ac0a5757d` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:28.034174 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8dd953-48a3-4d68-acc7-9b95ba878188 admin admin] RBAC: Authorizing `identity:delete_project(project_id=07331daf772d474b860f5f0ac0a5757d)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:28.036125 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8dd953-48a3-4d68-acc7-9b95ba878188 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:28.085646 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-0e8dd953-48a3-4d68-acc7-9b95ba878188 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07331daf772d474b860f5f0ac0a5757d', 'request_id': 'req-0e8dd953-48a3-4d68-acc7-9b95ba878188'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:28.086839 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:19:28.093441 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-0e8dd953-48a3-4d68-acc7-9b95ba878188 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07331daf772d474b860f5f0ac0a5757d', 'request_id': 'req-0e8dd953-48a3-4d68-acc7-9b95ba878188'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:28.103278 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-0e8dd953-48a3-4d68-acc7-9b95ba878188 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07331daf772d474b860f5f0ac0a5757d', 'request_id': 'req-0e8dd953-48a3-4d68-acc7-9b95ba878188'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:28.115939 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-0e8dd953-48a3-4d68-acc7-9b95ba878188 admin admin] The token cache is being invalidate because project 07331daf772d474b860f5f0ac0a5757d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:19:28.115939 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-0e8dd953-48a3-4d68-acc7-9b95ba878188 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:28.116618 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-0e8dd953-48a3-4d68-acc7-9b95ba878188 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:28.118446 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1269/2557] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 9 12:19:27 2022] DELETE /identity/v3/projects/07331daf772d474b860f5f0ac0a5757d => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:19:28.123165 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807dc5f4-b331-4d99-9286-3c30c7f96f11 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:28.123638 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807dc5f4-b331-4d99-9286-3c30c7f96f11 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:28.123722 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807dc5f4-b331-4d99-9286-3c30c7f96f11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:28.144656 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-807dc5f4-b331-4d99-9286-3c30c7f96f11 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:19:28.145988 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-807dc5f4-b331-4d99-9286-3c30c7f96f11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:19:28.218187 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1289/2558] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:19:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:19:28.226579 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9e30b1-0187-4dea-a945-02d3d59fd80a None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:28.227002 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9e30b1-0187-4dea-a945-02d3d59fd80a None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:28.227344 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9e30b1-0187-4dea-a945-02d3d59fd80a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:28.249876 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-aa9e30b1-0187-4dea-a945-02d3d59fd80a None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:19:28.251215 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-aa9e30b1-0187-4dea-a945-02d3d59fd80a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:19:28.263873 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1270/2559] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:19:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:19:28.265215 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807dc5f4-b331-4d99-9286-3c30c7f96f11 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:28.278085 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807dc5f4-b331-4d99-9286-3c30c7f96f11 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:28.285925 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d404a8-34ad-4a16-9910-db82f078b31a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:28.286755 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d404a8-34ad-4a16-9910-db82f078b31a admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:28.287065 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d404a8-34ad-4a16-9910-db82f078b31a admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:28.287339 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d404a8-34ad-4a16-9910-db82f078b31a admin admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:28.288878 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d404a8-34ad-4a16-9910-db82f078b31a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:19:28.289188 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d404a8-34ad-4a16-9910-db82f078b31a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:28.292091 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d404a8-34ad-4a16-9910-db82f078b31a admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:28.312520 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1290/2560] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 9 12:19:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:19:28.313630 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9e30b1-0187-4dea-a945-02d3d59fd80a None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:28.323749 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9e30b1-0187-4dea-a945-02d3d59fd80a None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:28.328578 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e354bda-4da5-436c-a209-b158312557fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:28.329392 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e354bda-4da5-436c-a209-b158312557fd admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:28.329733 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e354bda-4da5-436c-a209-b158312557fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:28.330064 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e354bda-4da5-436c-a209-b158312557fd admin admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:28.330755 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e354bda-4da5-436c-a209-b158312557fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:28.332724 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e354bda-4da5-436c-a209-b158312557fd admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:28.361457 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1271/2561] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 9 12:19:28 2022] POST /identity/v3/projects => generated 361 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:19:28.366282 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d404a8-34ad-4a16-9910-db82f078b31a admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:28.376788 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d404a8-34ad-4a16-9910-db82f078b31a admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:28.436855 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9939f039-b752-48df-8af4-df95e204e52b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:28.437466 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9939f039-b752-48df-8af4-df95e204e52b admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:28.437742 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9939f039-b752-48df-8af4-df95e204e52b admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:28.438011 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9939f039-b752-48df-8af4-df95e204e52b admin admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:28.438636 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9939f039-b752-48df-8af4-df95e204e52b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:28.440599 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9939f039-b752-48df-8af4-df95e204e52b admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:28.465085 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1291/2562] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:19:28 2022] POST /identity/v3/users => generated 281 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:19:28.468975 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e354bda-4da5-436c-a209-b158312557fd admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:28.481228 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e354bda-4da5-436c-a209-b158312557fd admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:28.486557 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2c5a36-600f-471f-98f1-21305bbdbb9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:28.487109 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2c5a36-600f-471f-98f1-21305bbdbb9b admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:28.487386 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2c5a36-600f-471f-98f1-21305bbdbb9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:28.487656 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2c5a36-600f-471f-98f1-21305bbdbb9b admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:28.488299 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2c5a36-600f-471f-98f1-21305bbdbb9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:28.489897 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2c5a36-600f-471f-98f1-21305bbdbb9b admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:28.502980 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1272/2563] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:19:28 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:19:28.506315 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9939f039-b752-48df-8af4-df95e204e52b admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:28.517088 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9939f039-b752-48df-8af4-df95e204e52b admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:28.521845 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475a5a7e-fb58-4646-98aa-a8d226b723b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:28.522461 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475a5a7e-fb58-4646-98aa-a8d226b723b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:28.522740 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475a5a7e-fb58-4646-98aa-a8d226b723b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:28.523047 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475a5a7e-fb58-4646-98aa-a8d226b723b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:28.523726 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475a5a7e-fb58-4646-98aa-a8d226b723b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:28.525392 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475a5a7e-fb58-4646-98aa-a8d226b723b9 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:28.536858 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1292/2564] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:19:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:19:28.542150 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2c5a36-600f-471f-98f1-21305bbdbb9b admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:28.553081 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2c5a36-600f-471f-98f1-21305bbdbb9b admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:28.559233 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcaf51b-4d0c-462b-af10-d67dc718ed3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:28.559906 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcaf51b-4d0c-462b-af10-d67dc718ed3d admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:28.560183 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcaf51b-4d0c-462b-af10-d67dc718ed3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:28.560452 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcaf51b-4d0c-462b-af10-d67dc718ed3d admin admin] PATH_INFO: `/v3/projects/71689619490843ac8acc4262859b190e/users/1e887bbb53cf4dbaafd08abb73f9fd30/roles/eecd2f160dd844f5906f3d345780c501` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:28.581240 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcaf51b-4d0c-462b-af10-d67dc718ed3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=71689619490843ac8acc4262859b190e, user_id=1e887bbb53cf4dbaafd08abb73f9fd30, role_id=eecd2f160dd844f5906f3d345780c501)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:28.584066 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcaf51b-4d0c-462b-af10-d67dc718ed3d admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:28.598716 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1273/2565] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:19:28 2022] PUT /identity/v3/projects/71689619490843ac8acc4262859b190e/users/1e887bbb53cf4dbaafd08abb73f9fd30/roles/eecd2f160dd844f5906f3d345780c501 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:19:28.610500 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d4c2ff-7613-4494-be62-15f7b90bfb10 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:28.611044 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d4c2ff-7613-4494-be62-15f7b90bfb10 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:28.611284 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d4c2ff-7613-4494-be62-15f7b90bfb10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:28.642337 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-55d4c2ff-7613-4494-be62-15f7b90bfb10 None None] MFA Rules not processed for user `1e887bbb53cf4dbaafd08abb73f9fd30`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:19:28.643717 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-55d4c2ff-7613-4494-be62-15f7b90bfb10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:19:28.686702 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:19:28.713863 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1293/2566] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:19:28 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:19:28.750343 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcaf51b-4d0c-462b-af10-d67dc718ed3d admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:28.752096 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-9dcaf51b-4d0c-462b-af10-d67dc718ed3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:19:28.817338 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcaf51b-4d0c-462b-af10-d67dc718ed3d admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:28.822004 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f0f285-7bd9-4602-8665-f1e19586eac4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535c258100df40b58a5ee44b2c21d621', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:28.822613 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f0f285-7bd9-4602-8665-f1e19586eac4 service cinder] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:28.823044 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f0f285-7bd9-4602-8665-f1e19586eac4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:28.823322 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f0f285-7bd9-4602-8665-f1e19586eac4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:28.832447 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f0f285-7bd9-4602-8665-f1e19586eac4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:28.833906 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f0f285-7bd9-4602-8665-f1e19586eac4 service cinder] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:28.842996 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1274/2567] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:19:28 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:19:28.859717 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d4c2ff-7613-4494-be62-15f7b90bfb10 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:28.928929 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d4c2ff-7613-4494-be62-15f7b90bfb10 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:28.933504 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc941ff8-4acf-4723-b49a-20ad110c1bff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:28.934137 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc941ff8-4acf-4723-b49a-20ad110c1bff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:28.934413 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc941ff8-4acf-4723-b49a-20ad110c1bff admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:28.934689 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc941ff8-4acf-4723-b49a-20ad110c1bff admin admin] PATH_INFO: `/v3/users/1e887bbb53cf4dbaafd08abb73f9fd30` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:28.935866 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc941ff8-4acf-4723-b49a-20ad110c1bff admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e887bbb53cf4dbaafd08abb73f9fd30)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:28.937785 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc941ff8-4acf-4723-b49a-20ad110c1bff admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:28.997170 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-bc941ff8-4acf-4723-b49a-20ad110c1bff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e887bbb53cf4dbaafd08abb73f9fd30'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:29.007338 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-bc941ff8-4acf-4723-b49a-20ad110c1bff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e887bbb53cf4dbaafd08abb73f9fd30'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:29.020038 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-bc941ff8-4acf-4723-b49a-20ad110c1bff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e887bbb53cf4dbaafd08abb73f9fd30'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:29.028409 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-bc941ff8-4acf-4723-b49a-20ad110c1bff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e887bbb53cf4dbaafd08abb73f9fd30'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:29.029174 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-bc941ff8-4acf-4723-b49a-20ad110c1bff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e887bbb53cf4dbaafd08abb73f9fd30'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:29.035495 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1294/2568] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 9 12:19:28 2022] DELETE /identity/v3/users/1e887bbb53cf4dbaafd08abb73f9fd30 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:19:29.040706 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61adbd7-ba62-4f7a-8df7-4b0d27f1d426 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:29.040933 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61adbd7-ba62-4f7a-8df7-4b0d27f1d426 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:29.041207 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61adbd7-ba62-4f7a-8df7-4b0d27f1d426 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:29.061815 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-b61adbd7-ba62-4f7a-8df7-4b0d27f1d426 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:19:29.063095 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-b61adbd7-ba62-4f7a-8df7-4b0d27f1d426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:19:29.125546 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1275/2569] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:19:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:19:29.135649 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc941ff8-4acf-4723-b49a-20ad110c1bff admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:29.137123 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-bc941ff8-4acf-4723-b49a-20ad110c1bff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:19:29.198130 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc941ff8-4acf-4723-b49a-20ad110c1bff admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:29.201777 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d340cbb7-29f1-4c69-a272-62f8f27c65ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:29.202477 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d340cbb7-29f1-4c69-a272-62f8f27c65ec service neutron] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:29.202820 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d340cbb7-29f1-4c69-a272-62f8f27c65ec service neutron] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:29.203143 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d340cbb7-29f1-4c69-a272-62f8f27c65ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:29.213946 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d340cbb7-29f1-4c69-a272-62f8f27c65ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:29.215465 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d340cbb7-29f1-4c69-a272-62f8f27c65ec service neutron] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:29.225533 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1295/2570] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:19:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:19:29.511973 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61adbd7-ba62-4f7a-8df7-4b0d27f1d426 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:29.513911 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-b61adbd7-ba62-4f7a-8df7-4b0d27f1d426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:19:29.529710 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61adbd7-ba62-4f7a-8df7-4b0d27f1d426 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:29.534607 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3735cc-5c86-4e8e-8452-540847101d39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:29.535321 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3735cc-5c86-4e8e-8452-540847101d39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:29.535602 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3735cc-5c86-4e8e-8452-540847101d39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:29.535923 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3735cc-5c86-4e8e-8452-540847101d39 admin admin] PATH_INFO: `/v3/projects/71689619490843ac8acc4262859b190e` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:29.537097 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3735cc-5c86-4e8e-8452-540847101d39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=71689619490843ac8acc4262859b190e)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:29.539780 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3735cc-5c86-4e8e-8452-540847101d39 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:29.552834 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:19:29.595671 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-8e3735cc-5c86-4e8e-8452-540847101d39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71689619490843ac8acc4262859b190e', 'request_id': 'req-8e3735cc-5c86-4e8e-8452-540847101d39'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:29.601946 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-8e3735cc-5c86-4e8e-8452-540847101d39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71689619490843ac8acc4262859b190e', 'request_id': 'req-8e3735cc-5c86-4e8e-8452-540847101d39'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:29.609358 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-8e3735cc-5c86-4e8e-8452-540847101d39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71689619490843ac8acc4262859b190e', 'request_id': 'req-8e3735cc-5c86-4e8e-8452-540847101d39'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:29.621330 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-8e3735cc-5c86-4e8e-8452-540847101d39 admin admin] The token cache is being invalidate because project 71689619490843ac8acc4262859b190e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:19:29.621619 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-8e3735cc-5c86-4e8e-8452-540847101d39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:29.622238 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-8e3735cc-5c86-4e8e-8452-540847101d39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:29.624101 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1276/2571] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 9 12:19:29 2022] DELETE /identity/v3/projects/71689619490843ac8acc4262859b190e => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:19:33.174934 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d340cbb7-29f1-4c69-a272-62f8f27c65ec service neutron] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:33.179252 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:19:33.185195 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d340cbb7-29f1-4c69-a272-62f8f27c65ec service neutron] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:33.235310 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b356e79d-10ba-436f-a935-9926f4a302eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:33.235898 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b356e79d-10ba-436f-a935-9926f4a302eb service neutron] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:33.236174 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b356e79d-10ba-436f-a935-9926f4a302eb service neutron] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:33.236441 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b356e79d-10ba-436f-a935-9926f4a302eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:33.238352 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-b356e79d-10ba-436f-a935-9926f4a302eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:19:33.300060 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b356e79d-10ba-436f-a935-9926f4a302eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:33.301767 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b356e79d-10ba-436f-a935-9926f4a302eb service neutron] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:33.312910 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1296/2572] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:19:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:19:36.481441 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3735cc-5c86-4e8e-8452-540847101d39 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:36.482940 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-8e3735cc-5c86-4e8e-8452-540847101d39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:19:36.497858 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3735cc-5c86-4e8e-8452-540847101d39 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:36.502335 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90356a7-d3b0-4f4e-a46e-0236d1c18d78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:36.502988 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90356a7-d3b0-4f4e-a46e-0236d1c18d78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:36.503263 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90356a7-d3b0-4f4e-a46e-0236d1c18d78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:36.503529 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90356a7-d3b0-4f4e-a46e-0236d1c18d78 admin admin] PATH_INFO: `/v3/users/2e125f66d9c84e9797099065351a07da` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:36.504636 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90356a7-d3b0-4f4e-a46e-0236d1c18d78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e125f66d9c84e9797099065351a07da)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:36.507641 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90356a7-d3b0-4f4e-a46e-0236d1c18d78 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:36.558935 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-b90356a7-d3b0-4f4e-a46e-0236d1c18d78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e125f66d9c84e9797099065351a07da'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:36.571221 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-b90356a7-d3b0-4f4e-a46e-0236d1c18d78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e125f66d9c84e9797099065351a07da'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:36.578050 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-b90356a7-d3b0-4f4e-a46e-0236d1c18d78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e125f66d9c84e9797099065351a07da'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:36.586294 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-b90356a7-d3b0-4f4e-a46e-0236d1c18d78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e125f66d9c84e9797099065351a07da'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:36.586991 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-b90356a7-d3b0-4f4e-a46e-0236d1c18d78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e125f66d9c84e9797099065351a07da'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:36.593230 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1277/2573] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 9 12:19:36 2022] DELETE /identity/v3/users/2e125f66d9c84e9797099065351a07da => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:19:36.773347 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b356e79d-10ba-436f-a935-9926f4a302eb service neutron] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:36.784156 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b356e79d-10ba-436f-a935-9926f4a302eb service neutron] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:36.841649 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f313088-eb24-4c48-a709-eecfa3a50ad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:36.842278 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f313088-eb24-4c48-a709-eecfa3a50ad3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:36.842556 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f313088-eb24-4c48-a709-eecfa3a50ad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:36.842829 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f313088-eb24-4c48-a709-eecfa3a50ad3 admin admin] PATH_INFO: `/v3/projects/30fa27304a4f4d4ab05c7763bc346d57` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:36.843978 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f313088-eb24-4c48-a709-eecfa3a50ad3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=30fa27304a4f4d4ab05c7763bc346d57)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:36.845870 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f313088-eb24-4c48-a709-eecfa3a50ad3 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:36.891844 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-6f313088-eb24-4c48-a709-eecfa3a50ad3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30fa27304a4f4d4ab05c7763bc346d57', 'request_id': 'req-6f313088-eb24-4c48-a709-eecfa3a50ad3'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:36.895605 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:19:36.898343 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-6f313088-eb24-4c48-a709-eecfa3a50ad3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30fa27304a4f4d4ab05c7763bc346d57', 'request_id': 'req-6f313088-eb24-4c48-a709-eecfa3a50ad3'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:36.904912 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-6f313088-eb24-4c48-a709-eecfa3a50ad3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30fa27304a4f4d4ab05c7763bc346d57', 'request_id': 'req-6f313088-eb24-4c48-a709-eecfa3a50ad3'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:36.917267 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-6f313088-eb24-4c48-a709-eecfa3a50ad3 admin admin] The token cache is being invalidate because project 30fa27304a4f4d4ab05c7763bc346d57 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81591) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:19:36.917580 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-6f313088-eb24-4c48-a709-eecfa3a50ad3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:36.918217 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-6f313088-eb24-4c48-a709-eecfa3a50ad3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:36.919815 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1297/2574] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 9 12:19:36 2022] DELETE /identity/v3/projects/30fa27304a4f4d4ab05c7763bc346d57 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:19:36.925013 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e392379d-e2b5-4b92-800b-85a69ee7f6cd None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:36.925316 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e392379d-e2b5-4b92-800b-85a69ee7f6cd None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:36.925593 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e392379d-e2b5-4b92-800b-85a69ee7f6cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:36.946509 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-e392379d-e2b5-4b92-800b-85a69ee7f6cd None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:19:36.947742 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-e392379d-e2b5-4b92-800b-85a69ee7f6cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:19:37.007402 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1278/2575] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:19:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:19:37.015353 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523bd5f9-dec3-4263-82ed-d2a9206b4f9c None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:37.015578 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523bd5f9-dec3-4263-82ed-d2a9206b4f9c None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:37.015852 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523bd5f9-dec3-4263-82ed-d2a9206b4f9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:37.035187 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-523bd5f9-dec3-4263-82ed-d2a9206b4f9c None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:19:37.036389 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-523bd5f9-dec3-4263-82ed-d2a9206b4f9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:19:37.044146 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1298/2576] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:19:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:19:37.048293 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e392379d-e2b5-4b92-800b-85a69ee7f6cd None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:37.058696 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e392379d-e2b5-4b92-800b-85a69ee7f6cd None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:37.063234 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3514e00e-e7fb-4010-88ea-3a20dde3252e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:37.063829 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3514e00e-e7fb-4010-88ea-3a20dde3252e admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:37.064103 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3514e00e-e7fb-4010-88ea-3a20dde3252e admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:37.064371 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3514e00e-e7fb-4010-88ea-3a20dde3252e admin admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:37.065001 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3514e00e-e7fb-4010-88ea-3a20dde3252e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:19:37.065299 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3514e00e-e7fb-4010-88ea-3a20dde3252e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:37.066878 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3514e00e-e7fb-4010-88ea-3a20dde3252e admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:37.076018 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:19:37.079364 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1279/2577] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 9 12:19:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:19:37.084022 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523bd5f9-dec3-4263-82ed-d2a9206b4f9c None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:37.093226 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523bd5f9-dec3-4263-82ed-d2a9206b4f9c None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:37.097910 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6708ef-a67f-4266-8633-1f381b6893c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:37.098498 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6708ef-a67f-4266-8633-1f381b6893c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:37.098776 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6708ef-a67f-4266-8633-1f381b6893c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:37.099080 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6708ef-a67f-4266-8633-1f381b6893c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:37.100053 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6708ef-a67f-4266-8633-1f381b6893c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:37.101934 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6708ef-a67f-4266-8633-1f381b6893c4 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:37.129353 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1299/2578] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 9 12:19:37 2022] POST /identity/v3/projects => generated 387 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:19:37.134424 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3514e00e-e7fb-4010-88ea-3a20dde3252e admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:37.144169 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3514e00e-e7fb-4010-88ea-3a20dde3252e admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:37.199406 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fdbb41-ea94-4e7a-99a9-2382234cf746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:37.199996 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fdbb41-ea94-4e7a-99a9-2382234cf746 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:37.200269 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fdbb41-ea94-4e7a-99a9-2382234cf746 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:37.200537 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fdbb41-ea94-4e7a-99a9-2382234cf746 admin admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:37.201148 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fdbb41-ea94-4e7a-99a9-2382234cf746 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:37.203081 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fdbb41-ea94-4e7a-99a9-2382234cf746 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:37.225170 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1280/2579] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:19:37 2022] POST /identity/v3/users => generated 294 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:19:37.229590 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6708ef-a67f-4266-8633-1f381b6893c4 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:37.239755 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6708ef-a67f-4266-8633-1f381b6893c4 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:37.244547 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527ce7fa-2f75-4d81-bf41-3a44aafda400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:37.245139 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527ce7fa-2f75-4d81-bf41-3a44aafda400 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:37.245418 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527ce7fa-2f75-4d81-bf41-3a44aafda400 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:37.245687 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527ce7fa-2f75-4d81-bf41-3a44aafda400 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:37.246353 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527ce7fa-2f75-4d81-bf41-3a44aafda400 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:37.248049 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527ce7fa-2f75-4d81-bf41-3a44aafda400 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:37.258592 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1300/2580] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:19:37 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:19:37.262478 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fdbb41-ea94-4e7a-99a9-2382234cf746 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:37.272997 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fdbb41-ea94-4e7a-99a9-2382234cf746 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:37.277457 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ed5b1d-50c2-4d16-ba3a-fa3e9531af7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:37.278038 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ed5b1d-50c2-4d16-ba3a-fa3e9531af7e admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:37.278311 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ed5b1d-50c2-4d16-ba3a-fa3e9531af7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:37.278580 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ed5b1d-50c2-4d16-ba3a-fa3e9531af7e admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:37.279245 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ed5b1d-50c2-4d16-ba3a-fa3e9531af7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:37.280822 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ed5b1d-50c2-4d16-ba3a-fa3e9531af7e admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:37.291281 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1281/2581] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:19:37 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:19:37.295636 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527ce7fa-2f75-4d81-bf41-3a44aafda400 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:37.305907 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527ce7fa-2f75-4d81-bf41-3a44aafda400 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:37.310527 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e65fd02-47f8-4943-b908-3d201ef1670d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:37.311221 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e65fd02-47f8-4943-b908-3d201ef1670d admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:37.311501 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e65fd02-47f8-4943-b908-3d201ef1670d admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:37.311772 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e65fd02-47f8-4943-b908-3d201ef1670d admin admin] PATH_INFO: `/v3/projects/4604ca91b34d4e5ebc6c3daec011eeaa/users/21ac20b179b24bd3857ecc1872b86dba/roles/eecd2f160dd844f5906f3d345780c501` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:37.326726 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e65fd02-47f8-4943-b908-3d201ef1670d admin admin] RBAC: Authorizing `identity:create_grant(project_id=4604ca91b34d4e5ebc6c3daec011eeaa, user_id=21ac20b179b24bd3857ecc1872b86dba, role_id=eecd2f160dd844f5906f3d345780c501)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:37.329396 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e65fd02-47f8-4943-b908-3d201ef1670d admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:37.342807 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1301/2582] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:19:37 2022] PUT /identity/v3/projects/4604ca91b34d4e5ebc6c3daec011eeaa/users/21ac20b179b24bd3857ecc1872b86dba/roles/eecd2f160dd844f5906f3d345780c501 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:19:37.353510 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72c124c-9911-4290-bc28-596154eeb4bc None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:37.353823 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72c124c-9911-4290-bc28-596154eeb4bc None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:37.354094 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72c124c-9911-4290-bc28-596154eeb4bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:37.381827 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-b72c124c-9911-4290-bc28-596154eeb4bc None None] MFA Rules not processed for user `21ac20b179b24bd3857ecc1872b86dba`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:19:37.383062 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-b72c124c-9911-4290-bc28-596154eeb4bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:19:37.445843 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1282/2583] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:19:37 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:19:37.450419 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e65fd02-47f8-4943-b908-3d201ef1670d admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:37.511134 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e65fd02-47f8-4943-b908-3d201ef1670d admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:37.515677 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801cd515-558e-4bcf-b38f-4e0e085cb525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:37.516297 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801cd515-558e-4bcf-b38f-4e0e085cb525 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:37.516574 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801cd515-558e-4bcf-b38f-4e0e085cb525 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:37.516843 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801cd515-558e-4bcf-b38f-4e0e085cb525 admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:37.517468 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801cd515-558e-4bcf-b38f-4e0e085cb525 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:37.519384 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801cd515-558e-4bcf-b38f-4e0e085cb525 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:37.545953 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1302/2584] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 9 12:19:37 2022] POST /identity/v3/projects => generated 387 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:19:37.549943 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72c124c-9911-4290-bc28-596154eeb4bc None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:37.560673 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72c124c-9911-4290-bc28-596154eeb4bc None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:37.616935 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46a8d53-b891-4103-93f8-b8ab23ab4589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:37.617531 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46a8d53-b891-4103-93f8-b8ab23ab4589 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:37.617807 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46a8d53-b891-4103-93f8-b8ab23ab4589 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:37.618073 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46a8d53-b891-4103-93f8-b8ab23ab4589 admin admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:37.618701 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46a8d53-b891-4103-93f8-b8ab23ab4589 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:37.620639 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46a8d53-b891-4103-93f8-b8ab23ab4589 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:37.636685 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:19:37.646647 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1283/2585] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:19:37 2022] POST /identity/v3/users => generated 300 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:19:37.650720 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801cd515-558e-4bcf-b38f-4e0e085cb525 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:37.660846 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801cd515-558e-4bcf-b38f-4e0e085cb525 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:37.666031 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af85bc79-a3f6-498c-b3bd-cce9ea669fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:37.666625 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af85bc79-a3f6-498c-b3bd-cce9ea669fb0 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:37.666923 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af85bc79-a3f6-498c-b3bd-cce9ea669fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:37.667201 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af85bc79-a3f6-498c-b3bd-cce9ea669fb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:37.667835 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af85bc79-a3f6-498c-b3bd-cce9ea669fb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:37.669445 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af85bc79-a3f6-498c-b3bd-cce9ea669fb0 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:37.680245 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1303/2586] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:19:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:19:37.684429 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46a8d53-b891-4103-93f8-b8ab23ab4589 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:37.694928 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46a8d53-b891-4103-93f8-b8ab23ab4589 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:37.699652 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3961079-ec33-4418-abc3-8d655e7e6e51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:37.700384 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3961079-ec33-4418-abc3-8d655e7e6e51 admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:37.700665 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3961079-ec33-4418-abc3-8d655e7e6e51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:37.700937 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3961079-ec33-4418-abc3-8d655e7e6e51 admin admin] PATH_INFO: `/v3/projects/007300a3dc4d4134a85f234b3d0144f1/users/8eda563332744d89858564a06026d601/roles/3f2ca33902a04b6dbc2ef7d06d7289d3` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:37.716422 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3961079-ec33-4418-abc3-8d655e7e6e51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=007300a3dc4d4134a85f234b3d0144f1, user_id=8eda563332744d89858564a06026d601, role_id=3f2ca33902a04b6dbc2ef7d06d7289d3)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:37.719133 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3961079-ec33-4418-abc3-8d655e7e6e51 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:37.732589 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1284/2587] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:19:37 2022] PUT /identity/v3/projects/007300a3dc4d4134a85f234b3d0144f1/users/8eda563332744d89858564a06026d601/roles/3f2ca33902a04b6dbc2ef7d06d7289d3 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:19:37.736681 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af85bc79-a3f6-498c-b3bd-cce9ea669fb0 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:37.746656 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af85bc79-a3f6-498c-b3bd-cce9ea669fb0 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:37.798171 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:19:37.810475 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680fcfd7-b8ee-44ca-b9c0-0dcefef6e5d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:37.811182 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680fcfd7-b8ee-44ca-b9c0-0dcefef6e5d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:37.811602 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680fcfd7-b8ee-44ca-b9c0-0dcefef6e5d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:37.811950 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680fcfd7-b8ee-44ca-b9c0-0dcefef6e5d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:37.812882 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680fcfd7-b8ee-44ca-b9c0-0dcefef6e5d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:37.814732 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680fcfd7-b8ee-44ca-b9c0-0dcefef6e5d5 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:37.825767 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1304/2588] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:19:37 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:19:37.830529 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3961079-ec33-4418-abc3-8d655e7e6e51 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:37.841928 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3961079-ec33-4418-abc3-8d655e7e6e51 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:37.847029 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9e4ba4-e741-4765-8962-908f08d5f48e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:37.847733 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9e4ba4-e741-4765-8962-908f08d5f48e admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:37.848038 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9e4ba4-e741-4765-8962-908f08d5f48e admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:37.848308 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9e4ba4-e741-4765-8962-908f08d5f48e admin admin] PATH_INFO: `/v3/projects/007300a3dc4d4134a85f234b3d0144f1/users/8eda563332744d89858564a06026d601/roles/3f2ca33902a04b6dbc2ef7d06d7289d3` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:37.850991 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9e4ba4-e741-4765-8962-908f08d5f48e admin admin] RBAC: Authorizing `identity:create_grant(project_id=007300a3dc4d4134a85f234b3d0144f1, user_id=8eda563332744d89858564a06026d601, role_id=3f2ca33902a04b6dbc2ef7d06d7289d3)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:37.853829 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9e4ba4-e741-4765-8962-908f08d5f48e admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:37.866313 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1285/2589] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:19:37 2022] PUT /identity/v3/projects/007300a3dc4d4134a85f234b3d0144f1/users/8eda563332744d89858564a06026d601/roles/3f2ca33902a04b6dbc2ef7d06d7289d3 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:19:37.878541 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a835cb13-0dee-4927-b301-c44e2e79df2c None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:37.878938 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a835cb13-0dee-4927-b301-c44e2e79df2c None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:37.879223 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a835cb13-0dee-4927-b301-c44e2e79df2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:37.909957 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-a835cb13-0dee-4927-b301-c44e2e79df2c None None] MFA Rules not processed for user `8eda563332744d89858564a06026d601`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:19:37.911223 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-a835cb13-0dee-4927-b301-c44e2e79df2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:19:37.980278 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1305/2590] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:19:37 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:19:37.991423 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9e4ba4-e741-4765-8962-908f08d5f48e admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:37.993123 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-3b9e4ba4-e741-4765-8962-908f08d5f48e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:19:38.049766 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9e4ba4-e741-4765-8962-908f08d5f48e admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:38.053836 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcae358-acb3-4df7-a056-c33499b2e683 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535c258100df40b58a5ee44b2c21d621', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:38.054445 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcae358-acb3-4df7-a056-c33499b2e683 service cinder] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:38.054756 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcae358-acb3-4df7-a056-c33499b2e683 service cinder] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:38.055059 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcae358-acb3-4df7-a056-c33499b2e683 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:38.064021 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcae358-acb3-4df7-a056-c33499b2e683 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:38.065456 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcae358-acb3-4df7-a056-c33499b2e683 service cinder] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:38.075628 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1286/2591] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:19:37 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:19:38.104438 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a835cb13-0dee-4927-b301-c44e2e79df2c None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:38.164894 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a835cb13-0dee-4927-b301-c44e2e79df2c None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:38.169474 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ab5e39-9917-4672-b78d-fec81ae5c15e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:38.170094 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ab5e39-9917-4672-b78d-fec81ae5c15e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:38.170373 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ab5e39-9917-4672-b78d-fec81ae5c15e admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:38.170650 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ab5e39-9917-4672-b78d-fec81ae5c15e admin admin] PATH_INFO: `/v3/users/21ac20b179b24bd3857ecc1872b86dba` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:38.171799 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ab5e39-9917-4672-b78d-fec81ae5c15e admin admin] RBAC: Authorizing `identity:delete_user(user_id=21ac20b179b24bd3857ecc1872b86dba)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:38.173705 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ab5e39-9917-4672-b78d-fec81ae5c15e admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:38.221438 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-45ab5e39-9917-4672-b78d-fec81ae5c15e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21ac20b179b24bd3857ecc1872b86dba'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:38.228653 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-45ab5e39-9917-4672-b78d-fec81ae5c15e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21ac20b179b24bd3857ecc1872b86dba'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:38.240174 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-45ab5e39-9917-4672-b78d-fec81ae5c15e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21ac20b179b24bd3857ecc1872b86dba'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:38.242631 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:19:38.247700 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-45ab5e39-9917-4672-b78d-fec81ae5c15e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21ac20b179b24bd3857ecc1872b86dba'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:38.248563 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-45ab5e39-9917-4672-b78d-fec81ae5c15e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21ac20b179b24bd3857ecc1872b86dba'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:38.255383 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1306/2592] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 9 12:19:38 2022] DELETE /identity/v3/users/21ac20b179b24bd3857ecc1872b86dba => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:19:38.260534 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fc7baa-d988-4182-85e8-00c4710abd63 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:38.260823 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fc7baa-d988-4182-85e8-00c4710abd63 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:38.261127 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fc7baa-d988-4182-85e8-00c4710abd63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:38.281675 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-c4fc7baa-d988-4182-85e8-00c4710abd63 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:19:38.282859 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-c4fc7baa-d988-4182-85e8-00c4710abd63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:19:38.340893 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1287/2593] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:19:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:19:38.350526 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ab5e39-9917-4672-b78d-fec81ae5c15e admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:38.352112 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-45ab5e39-9917-4672-b78d-fec81ae5c15e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:19:38.408997 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ab5e39-9917-4672-b78d-fec81ae5c15e admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:38.412835 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73577b83-070e-4ce7-9844-a811059bdf99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:38.413439 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73577b83-070e-4ce7-9844-a811059bdf99 service neutron] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:38.413719 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73577b83-070e-4ce7-9844-a811059bdf99 service neutron] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:38.413991 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73577b83-070e-4ce7-9844-a811059bdf99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:38.422863 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73577b83-070e-4ce7-9844-a811059bdf99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:38.424331 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73577b83-070e-4ce7-9844-a811059bdf99 service neutron] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:38.434440 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1307/2594] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:19:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:19:38.710141 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fc7baa-d988-4182-85e8-00c4710abd63 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:38.711620 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-c4fc7baa-d988-4182-85e8-00c4710abd63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:19:38.723478 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fc7baa-d988-4182-85e8-00c4710abd63 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:38.728321 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f26c624-2285-476c-a1b7-d285874a9fbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:38.728951 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f26c624-2285-476c-a1b7-d285874a9fbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:38.729222 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f26c624-2285-476c-a1b7-d285874a9fbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:38.729487 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f26c624-2285-476c-a1b7-d285874a9fbd admin admin] PATH_INFO: `/v3/projects/4604ca91b34d4e5ebc6c3daec011eeaa` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:38.730598 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f26c624-2285-476c-a1b7-d285874a9fbd admin admin] RBAC: Authorizing `identity:delete_project(project_id=4604ca91b34d4e5ebc6c3daec011eeaa)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:38.732538 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f26c624-2285-476c-a1b7-d285874a9fbd admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:38.780919 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-7f26c624-2285-476c-a1b7-d285874a9fbd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4604ca91b34d4e5ebc6c3daec011eeaa', 'request_id': 'req-7f26c624-2285-476c-a1b7-d285874a9fbd'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:38.786308 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-7f26c624-2285-476c-a1b7-d285874a9fbd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4604ca91b34d4e5ebc6c3daec011eeaa', 'request_id': 'req-7f26c624-2285-476c-a1b7-d285874a9fbd'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:38.788737 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:19:38.795076 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-7f26c624-2285-476c-a1b7-d285874a9fbd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4604ca91b34d4e5ebc6c3daec011eeaa', 'request_id': 'req-7f26c624-2285-476c-a1b7-d285874a9fbd'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:38.808529 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-7f26c624-2285-476c-a1b7-d285874a9fbd admin admin] The token cache is being invalidate because project 4604ca91b34d4e5ebc6c3daec011eeaa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:19:38.808811 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-7f26c624-2285-476c-a1b7-d285874a9fbd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:38.809454 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-7f26c624-2285-476c-a1b7-d285874a9fbd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:38.810948 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1288/2595] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 9 12:19:38 2022] DELETE /identity/v3/projects/4604ca91b34d4e5ebc6c3daec011eeaa => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:19:38.814940 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73577b83-070e-4ce7-9844-a811059bdf99 service neutron] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:38.824807 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73577b83-070e-4ce7-9844-a811059bdf99 service neutron] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:38.880587 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21244557-4138-4ca2-a59f-1a57c3017d6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:38.881280 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21244557-4138-4ca2-a59f-1a57c3017d6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:38.881559 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21244557-4138-4ca2-a59f-1a57c3017d6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:38.881832 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21244557-4138-4ca2-a59f-1a57c3017d6e admin admin] PATH_INFO: `/v3/users/8eda563332744d89858564a06026d601` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:38.882988 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21244557-4138-4ca2-a59f-1a57c3017d6e admin admin] RBAC: Authorizing `identity:delete_user(user_id=8eda563332744d89858564a06026d601)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:38.884902 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21244557-4138-4ca2-a59f-1a57c3017d6e admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:38.929838 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-21244557-4138-4ca2-a59f-1a57c3017d6e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8eda563332744d89858564a06026d601'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:38.936539 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-21244557-4138-4ca2-a59f-1a57c3017d6e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8eda563332744d89858564a06026d601'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:38.947909 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-21244557-4138-4ca2-a59f-1a57c3017d6e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8eda563332744d89858564a06026d601'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:38.955260 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-21244557-4138-4ca2-a59f-1a57c3017d6e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8eda563332744d89858564a06026d601'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:38.955992 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-21244557-4138-4ca2-a59f-1a57c3017d6e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8eda563332744d89858564a06026d601'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:38.961955 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1308/2596] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 9 12:19:38 2022] DELETE /identity/v3/users/8eda563332744d89858564a06026d601 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:19:39.234121 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f26c624-2285-476c-a1b7-d285874a9fbd admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:39.235560 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-7f26c624-2285-476c-a1b7-d285874a9fbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:19:39.248744 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f26c624-2285-476c-a1b7-d285874a9fbd admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:39.306644 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f03a9a7-fa3f-4509-8a62-813dc2f994f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:39.307314 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f03a9a7-fa3f-4509-8a62-813dc2f994f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:39.307593 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f03a9a7-fa3f-4509-8a62-813dc2f994f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:39.307865 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f03a9a7-fa3f-4509-8a62-813dc2f994f9 admin admin] PATH_INFO: `/v3/projects/007300a3dc4d4134a85f234b3d0144f1` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:39.308983 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f03a9a7-fa3f-4509-8a62-813dc2f994f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=007300a3dc4d4134a85f234b3d0144f1)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:39.310915 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f03a9a7-fa3f-4509-8a62-813dc2f994f9 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:39.357259 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-7f03a9a7-fa3f-4509-8a62-813dc2f994f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '007300a3dc4d4134a85f234b3d0144f1', 'request_id': 'req-7f03a9a7-fa3f-4509-8a62-813dc2f994f9'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:39.362671 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-7f03a9a7-fa3f-4509-8a62-813dc2f994f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '007300a3dc4d4134a85f234b3d0144f1', 'request_id': 'req-7f03a9a7-fa3f-4509-8a62-813dc2f994f9'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:39.369897 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-7f03a9a7-fa3f-4509-8a62-813dc2f994f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '007300a3dc4d4134a85f234b3d0144f1', 'request_id': 'req-7f03a9a7-fa3f-4509-8a62-813dc2f994f9'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:39.380917 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-7f03a9a7-fa3f-4509-8a62-813dc2f994f9 admin admin] The token cache is being invalidate because project 007300a3dc4d4134a85f234b3d0144f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:19:39.381206 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-7f03a9a7-fa3f-4509-8a62-813dc2f994f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:39.381838 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-7f03a9a7-fa3f-4509-8a62-813dc2f994f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:39.383349 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1289/2597] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 9 12:19:39 2022] DELETE /identity/v3/projects/007300a3dc4d4134a85f234b3d0144f1 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:19:39.388361 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58689796-5259-442b-9e06-4506a81da98f None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:39.388650 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58689796-5259-442b-9e06-4506a81da98f None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:39.388930 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58689796-5259-442b-9e06-4506a81da98f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:39.409045 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-58689796-5259-442b-9e06-4506a81da98f None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:19:39.410275 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-58689796-5259-442b-9e06-4506a81da98f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:19:39.420011 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:19:39.471558 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1309/2598] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:19:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:19:39.479901 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b8e25a-82ec-4f0b-84ae-75c475020192 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:39.480185 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b8e25a-82ec-4f0b-84ae-75c475020192 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:39.480489 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b8e25a-82ec-4f0b-84ae-75c475020192 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:39.500175 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-85b8e25a-82ec-4f0b-84ae-75c475020192 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:19:39.501406 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-85b8e25a-82ec-4f0b-84ae-75c475020192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:19:39.509845 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1290/2599] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:19:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:19:39.514069 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58689796-5259-442b-9e06-4506a81da98f None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:39.524202 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58689796-5259-442b-9e06-4506a81da98f None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:39.528909 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3e767f-4554-4d7a-8cd9-5e13afd995c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:39.529504 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3e767f-4554-4d7a-8cd9-5e13afd995c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:39.529779 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3e767f-4554-4d7a-8cd9-5e13afd995c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:39.530045 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3e767f-4554-4d7a-8cd9-5e13afd995c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:39.530687 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3e767f-4554-4d7a-8cd9-5e13afd995c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:19:39.531021 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3e767f-4554-4d7a-8cd9-5e13afd995c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:39.532636 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3e767f-4554-4d7a-8cd9-5e13afd995c7 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:39.543992 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1310/2600] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 9 12:19:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:19:39.548239 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b8e25a-82ec-4f0b-84ae-75c475020192 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:39.557614 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b8e25a-82ec-4f0b-84ae-75c475020192 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:39.562264 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83155596-d5bd-45df-831b-bf8a7fef645f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:39.562853 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83155596-d5bd-45df-831b-bf8a7fef645f admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:39.563164 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83155596-d5bd-45df-831b-bf8a7fef645f admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:39.563433 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83155596-d5bd-45df-831b-bf8a7fef645f admin admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:39.564064 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83155596-d5bd-45df-831b-bf8a7fef645f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:39.566004 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83155596-d5bd-45df-831b-bf8a7fef645f admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:39.593903 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1291/2601] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 9 12:19:39 2022] POST /identity/v3/projects => generated 389 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:19:39.598046 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3e767f-4554-4d7a-8cd9-5e13afd995c7 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:39.607960 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3e767f-4554-4d7a-8cd9-5e13afd995c7 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:39.666223 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abea954-7f15-4e4c-a17b-bd3a2a58baea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:39.666767 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abea954-7f15-4e4c-a17b-bd3a2a58baea admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:39.667083 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abea954-7f15-4e4c-a17b-bd3a2a58baea admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:39.667358 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abea954-7f15-4e4c-a17b-bd3a2a58baea admin admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:39.668000 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abea954-7f15-4e4c-a17b-bd3a2a58baea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:39.669985 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abea954-7f15-4e4c-a17b-bd3a2a58baea admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:39.692209 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1311/2602] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:19:39 2022] POST /identity/v3/users => generated 295 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:19:39.696980 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83155596-d5bd-45df-831b-bf8a7fef645f admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:39.707987 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83155596-d5bd-45df-831b-bf8a7fef645f admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:39.713231 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9fa0de-a591-4ada-b135-8ccf999d79a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:39.713820 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9fa0de-a591-4ada-b135-8ccf999d79a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:39.714099 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9fa0de-a591-4ada-b135-8ccf999d79a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:39.714369 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9fa0de-a591-4ada-b135-8ccf999d79a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:39.715048 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9fa0de-a591-4ada-b135-8ccf999d79a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:39.716645 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9fa0de-a591-4ada-b135-8ccf999d79a1 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:39.726551 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1292/2603] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:19:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:19:39.730502 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abea954-7f15-4e4c-a17b-bd3a2a58baea admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:39.740784 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abea954-7f15-4e4c-a17b-bd3a2a58baea admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:39.745308 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5c5c77-14e6-4816-8a83-bbacddd888e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:39.745902 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5c5c77-14e6-4816-8a83-bbacddd888e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:39.746181 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5c5c77-14e6-4816-8a83-bbacddd888e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:39.746456 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5c5c77-14e6-4816-8a83-bbacddd888e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:39.747124 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5c5c77-14e6-4816-8a83-bbacddd888e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:39.748712 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5c5c77-14e6-4816-8a83-bbacddd888e3 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:39.760099 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1312/2604] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:19:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 12:19:39.764198 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9fa0de-a591-4ada-b135-8ccf999d79a1 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:39.775042 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9fa0de-a591-4ada-b135-8ccf999d79a1 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:39.779801 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505e6d03-3395-47ca-8c7c-00eee70efa4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:39.780514 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505e6d03-3395-47ca-8c7c-00eee70efa4e admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:39.780928 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505e6d03-3395-47ca-8c7c-00eee70efa4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:39.781222 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505e6d03-3395-47ca-8c7c-00eee70efa4e admin admin] PATH_INFO: `/v3/projects/b0a544ec11684963b3fe297e057cd036/users/c0d6df1d824c4cb993490f872e7bcf2a/roles/eecd2f160dd844f5906f3d345780c501` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:39.796047 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505e6d03-3395-47ca-8c7c-00eee70efa4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0a544ec11684963b3fe297e057cd036, user_id=c0d6df1d824c4cb993490f872e7bcf2a, role_id=eecd2f160dd844f5906f3d345780c501)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:39.798711 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505e6d03-3395-47ca-8c7c-00eee70efa4e admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:39.812621 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1293/2605] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:19:39 2022] PUT /identity/v3/projects/b0a544ec11684963b3fe297e057cd036/users/c0d6df1d824c4cb993490f872e7bcf2a/roles/eecd2f160dd844f5906f3d345780c501 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:19:39.824536 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47efa9c-77bf-4884-98cd-5b17b05aafe9 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:39.824973 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47efa9c-77bf-4884-98cd-5b17b05aafe9 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:39.825091 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47efa9c-77bf-4884-98cd-5b17b05aafe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:39.853678 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-f47efa9c-77bf-4884-98cd-5b17b05aafe9 None None] MFA Rules not processed for user `c0d6df1d824c4cb993490f872e7bcf2a`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:19:39.854866 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-f47efa9c-77bf-4884-98cd-5b17b05aafe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:19:39.917594 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1313/2606] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:19:39 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:19:39.922045 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505e6d03-3395-47ca-8c7c-00eee70efa4e admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:39.936631 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:INSERT=4,keystone:DELETE=8,keystone:SELECT=88 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:19:39.985849 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505e6d03-3395-47ca-8c7c-00eee70efa4e admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:39.991218 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9240379-d737-43f0-a982-93fd07c4e9cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:39.991817 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9240379-d737-43f0-a982-93fd07c4e9cc admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:39.992087 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9240379-d737-43f0-a982-93fd07c4e9cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:39.992351 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9240379-d737-43f0-a982-93fd07c4e9cc admin admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:39.992968 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9240379-d737-43f0-a982-93fd07c4e9cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:39.994871 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9240379-d737-43f0-a982-93fd07c4e9cc admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:40.021130 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1294/2607] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 9 12:19:39 2022] POST /identity/v3/projects => generated 389 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:19:40.025560 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47efa9c-77bf-4884-98cd-5b17b05aafe9 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:40.036154 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47efa9c-77bf-4884-98cd-5b17b05aafe9 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:40.043782 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:19:40.093811 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0228a7e-3b44-445b-b737-aa697f1f89de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:40.094416 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0228a7e-3b44-445b-b737-aa697f1f89de admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:40.094690 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0228a7e-3b44-445b-b737-aa697f1f89de admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:40.094988 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0228a7e-3b44-445b-b737-aa697f1f89de admin admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:40.095609 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0228a7e-3b44-445b-b737-aa697f1f89de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:40.097537 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0228a7e-3b44-445b-b737-aa697f1f89de admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:40.117934 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1314/2608] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:19:40 2022] POST /identity/v3/users => generated 301 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:19:40.121967 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9240379-d737-43f0-a982-93fd07c4e9cc admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:40.133091 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9240379-d737-43f0-a982-93fd07c4e9cc admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:40.137738 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17026dda-2580-43f2-961b-eb535217e98e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:40.138324 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17026dda-2580-43f2-961b-eb535217e98e admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:40.138598 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17026dda-2580-43f2-961b-eb535217e98e admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:40.138864 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17026dda-2580-43f2-961b-eb535217e98e admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:40.139525 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17026dda-2580-43f2-961b-eb535217e98e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:40.141107 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17026dda-2580-43f2-961b-eb535217e98e admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:40.151652 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1295/2609] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:19:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:19:40.155753 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0228a7e-3b44-445b-b737-aa697f1f89de admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:40.165632 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0228a7e-3b44-445b-b737-aa697f1f89de admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:40.170261 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8bf40a-3b26-493a-808a-139379b56426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:40.170845 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8bf40a-3b26-493a-808a-139379b56426 admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:40.171164 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8bf40a-3b26-493a-808a-139379b56426 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:40.171442 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8bf40a-3b26-493a-808a-139379b56426 admin admin] PATH_INFO: `/v3/projects/d4b444984a704a4485618f3a93ce95a0/users/11d42cacea274426aa85da14c8ee6202/roles/3f2ca33902a04b6dbc2ef7d06d7289d3` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:40.186587 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8bf40a-3b26-493a-808a-139379b56426 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4b444984a704a4485618f3a93ce95a0, user_id=11d42cacea274426aa85da14c8ee6202, role_id=3f2ca33902a04b6dbc2ef7d06d7289d3)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:40.189294 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8bf40a-3b26-493a-808a-139379b56426 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:40.202365 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1315/2610] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:19:40 2022] PUT /identity/v3/projects/d4b444984a704a4485618f3a93ce95a0/users/11d42cacea274426aa85da14c8ee6202/roles/3f2ca33902a04b6dbc2ef7d06d7289d3 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:19:40.206860 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17026dda-2580-43f2-961b-eb535217e98e admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:40.217288 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17026dda-2580-43f2-961b-eb535217e98e admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:40.273028 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a163e020-e83d-483c-bee1-2631a49fd27b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:40.273633 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a163e020-e83d-483c-bee1-2631a49fd27b admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:40.273911 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a163e020-e83d-483c-bee1-2631a49fd27b admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:40.274183 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a163e020-e83d-483c-bee1-2631a49fd27b admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:40.274846 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a163e020-e83d-483c-bee1-2631a49fd27b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:40.276473 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a163e020-e83d-483c-bee1-2631a49fd27b admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:40.286415 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1296/2611] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:19:40 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:19:40.290795 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8bf40a-3b26-493a-808a-139379b56426 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:40.300869 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8bf40a-3b26-493a-808a-139379b56426 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:40.306298 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d858f8-e3b4-41f2-b25a-bbcb7bdd2235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:40.306973 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d858f8-e3b4-41f2-b25a-bbcb7bdd2235 admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:40.307256 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d858f8-e3b4-41f2-b25a-bbcb7bdd2235 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:40.307532 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d858f8-e3b4-41f2-b25a-bbcb7bdd2235 admin admin] PATH_INFO: `/v3/projects/d4b444984a704a4485618f3a93ce95a0/users/11d42cacea274426aa85da14c8ee6202/roles/3f2ca33902a04b6dbc2ef7d06d7289d3` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:40.309457 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d858f8-e3b4-41f2-b25a-bbcb7bdd2235 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4b444984a704a4485618f3a93ce95a0, user_id=11d42cacea274426aa85da14c8ee6202, role_id=3f2ca33902a04b6dbc2ef7d06d7289d3)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:40.312130 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d858f8-e3b4-41f2-b25a-bbcb7bdd2235 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:40.322423 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1316/2612] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:19:40 2022] PUT /identity/v3/projects/d4b444984a704a4485618f3a93ce95a0/users/11d42cacea274426aa85da14c8ee6202/roles/3f2ca33902a04b6dbc2ef7d06d7289d3 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:19:40.333758 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fea84f-14d6-43fd-b237-c158aa45891d None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:40.334030 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fea84f-14d6-43fd-b237-c158aa45891d None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:40.334297 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fea84f-14d6-43fd-b237-c158aa45891d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:40.362052 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-31fea84f-14d6-43fd-b237-c158aa45891d None None] MFA Rules not processed for user `11d42cacea274426aa85da14c8ee6202`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:19:40.363421 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-31fea84f-14d6-43fd-b237-c158aa45891d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:19:40.432908 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1297/2613] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:19:40 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:19:40.443554 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d858f8-e3b4-41f2-b25a-bbcb7bdd2235 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:40.445266 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-f7d858f8-e3b4-41f2-b25a-bbcb7bdd2235 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:19:40.501416 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d858f8-e3b4-41f2-b25a-bbcb7bdd2235 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:40.505001 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90c1247-9094-4c51-a9f2-329536e8417e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535c258100df40b58a5ee44b2c21d621', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:40.506204 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90c1247-9094-4c51-a9f2-329536e8417e service cinder] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:40.506478 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90c1247-9094-4c51-a9f2-329536e8417e service cinder] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:40.506744 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90c1247-9094-4c51-a9f2-329536e8417e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:40.515555 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90c1247-9094-4c51-a9f2-329536e8417e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:40.516969 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90c1247-9094-4c51-a9f2-329536e8417e service cinder] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:40.526719 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1317/2614] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:19:40 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:19:40.663547 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fea84f-14d6-43fd-b237-c158aa45891d None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:40.673003 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:19:40.725972 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fea84f-14d6-43fd-b237-c158aa45891d None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:40.730351 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0496ca4-a537-4ff7-ac51-f86b514fc785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:40.731001 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0496ca4-a537-4ff7-ac51-f86b514fc785 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:40.731281 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0496ca4-a537-4ff7-ac51-f86b514fc785 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:40.731552 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0496ca4-a537-4ff7-ac51-f86b514fc785 admin admin] PATH_INFO: `/v3/users/c0d6df1d824c4cb993490f872e7bcf2a` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:40.732670 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0496ca4-a537-4ff7-ac51-f86b514fc785 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0d6df1d824c4cb993490f872e7bcf2a)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:40.734614 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0496ca4-a537-4ff7-ac51-f86b514fc785 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:40.781026 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-b0496ca4-a537-4ff7-ac51-f86b514fc785 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0d6df1d824c4cb993490f872e7bcf2a'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:40.794149 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-b0496ca4-a537-4ff7-ac51-f86b514fc785 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0d6df1d824c4cb993490f872e7bcf2a'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:40.801014 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-b0496ca4-a537-4ff7-ac51-f86b514fc785 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0d6df1d824c4cb993490f872e7bcf2a'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:40.808593 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-b0496ca4-a537-4ff7-ac51-f86b514fc785 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0d6df1d824c4cb993490f872e7bcf2a'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:40.809233 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-b0496ca4-a537-4ff7-ac51-f86b514fc785 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0d6df1d824c4cb993490f872e7bcf2a'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:40.815274 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1298/2615] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 9 12:19:40 2022] DELETE /identity/v3/users/c0d6df1d824c4cb993490f872e7bcf2a => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:19:40.820505 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab713cd6-690a-4cc2-ad7b-d1867b79037e None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:40.820793 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab713cd6-690a-4cc2-ad7b-d1867b79037e None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:40.821093 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab713cd6-690a-4cc2-ad7b-d1867b79037e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:40.839708 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-ab713cd6-690a-4cc2-ad7b-d1867b79037e None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:19:40.840916 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-ab713cd6-690a-4cc2-ad7b-d1867b79037e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:19:40.897850 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1318/2616] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:19:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:19:40.907712 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0496ca4-a537-4ff7-ac51-f86b514fc785 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:40.909150 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-b0496ca4-a537-4ff7-ac51-f86b514fc785 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:19:40.965760 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0496ca4-a537-4ff7-ac51-f86b514fc785 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:40.969375 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dfeb7f-e95d-4a6d-9aa4-db30b80314a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:40.969973 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dfeb7f-e95d-4a6d-9aa4-db30b80314a9 service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:40.970248 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dfeb7f-e95d-4a6d-9aa4-db30b80314a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:40.970544 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dfeb7f-e95d-4a6d-9aa4-db30b80314a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:40.979802 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dfeb7f-e95d-4a6d-9aa4-db30b80314a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:40.981182 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dfeb7f-e95d-4a6d-9aa4-db30b80314a9 service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:40.991351 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1299/2617] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:19:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:19:41.262209 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab713cd6-690a-4cc2-ad7b-d1867b79037e None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:41.263817 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-ab713cd6-690a-4cc2-ad7b-d1867b79037e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:19:41.276559 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab713cd6-690a-4cc2-ad7b-d1867b79037e None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:41.281204 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c79c23-a332-4cf9-a5de-efbf991cef51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:41.282027 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c79c23-a332-4cf9-a5de-efbf991cef51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:41.282228 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c79c23-a332-4cf9-a5de-efbf991cef51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:41.282498 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c79c23-a332-4cf9-a5de-efbf991cef51 admin admin] PATH_INFO: `/v3/projects/b0a544ec11684963b3fe297e057cd036` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:41.283714 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c79c23-a332-4cf9-a5de-efbf991cef51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0a544ec11684963b3fe297e057cd036)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:41.285608 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c79c23-a332-4cf9-a5de-efbf991cef51 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:41.287436 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:19:41.334361 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-77c79c23-a332-4cf9-a5de-efbf991cef51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0a544ec11684963b3fe297e057cd036', 'request_id': 'req-77c79c23-a332-4cf9-a5de-efbf991cef51'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:41.339915 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-77c79c23-a332-4cf9-a5de-efbf991cef51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0a544ec11684963b3fe297e057cd036', 'request_id': 'req-77c79c23-a332-4cf9-a5de-efbf991cef51'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:41.345970 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-77c79c23-a332-4cf9-a5de-efbf991cef51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0a544ec11684963b3fe297e057cd036', 'request_id': 'req-77c79c23-a332-4cf9-a5de-efbf991cef51'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:41.358461 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-77c79c23-a332-4cf9-a5de-efbf991cef51 admin admin] The token cache is being invalidate because project b0a544ec11684963b3fe297e057cd036 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81591) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:19:41.358750 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-77c79c23-a332-4cf9-a5de-efbf991cef51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:41.359415 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-77c79c23-a332-4cf9-a5de-efbf991cef51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:41.361123 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1319/2618] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 9 12:19:41 2022] DELETE /identity/v3/projects/b0a544ec11684963b3fe297e057cd036 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:19:41.365418 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dfeb7f-e95d-4a6d-9aa4-db30b80314a9 service neutron] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:41.375047 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dfeb7f-e95d-4a6d-9aa4-db30b80314a9 service neutron] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:41.403928 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:19:41.433545 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ad0c06-b17e-47dd-b2e8-e4668c3206d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:41.434169 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ad0c06-b17e-47dd-b2e8-e4668c3206d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:41.434446 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ad0c06-b17e-47dd-b2e8-e4668c3206d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:41.434714 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ad0c06-b17e-47dd-b2e8-e4668c3206d8 admin admin] PATH_INFO: `/v3/users/11d42cacea274426aa85da14c8ee6202` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:41.435862 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ad0c06-b17e-47dd-b2e8-e4668c3206d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=11d42cacea274426aa85da14c8ee6202)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:41.438003 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ad0c06-b17e-47dd-b2e8-e4668c3206d8 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:41.484735 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-e7ad0c06-b17e-47dd-b2e8-e4668c3206d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11d42cacea274426aa85da14c8ee6202'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:41.497042 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-e7ad0c06-b17e-47dd-b2e8-e4668c3206d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11d42cacea274426aa85da14c8ee6202'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:41.503764 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-e7ad0c06-b17e-47dd-b2e8-e4668c3206d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11d42cacea274426aa85da14c8ee6202'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:41.511607 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-e7ad0c06-b17e-47dd-b2e8-e4668c3206d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11d42cacea274426aa85da14c8ee6202'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:41.512164 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-e7ad0c06-b17e-47dd-b2e8-e4668c3206d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11d42cacea274426aa85da14c8ee6202'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:41.517931 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1300/2619] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 9 12:19:41 2022] DELETE /identity/v3/users/11d42cacea274426aa85da14c8ee6202 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:19:41.788399 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c79c23-a332-4cf9-a5de-efbf991cef51 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:41.789835 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-77c79c23-a332-4cf9-a5de-efbf991cef51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:19:41.809982 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c79c23-a332-4cf9-a5de-efbf991cef51 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:41.866253 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34e0b06-1fb3-4f51-a1bb-8ed05d5d63ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:41.866950 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34e0b06-1fb3-4f51-a1bb-8ed05d5d63ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:41.867180 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34e0b06-1fb3-4f51-a1bb-8ed05d5d63ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:41.867452 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34e0b06-1fb3-4f51-a1bb-8ed05d5d63ed admin admin] PATH_INFO: `/v3/projects/d4b444984a704a4485618f3a93ce95a0` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:41.868570 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34e0b06-1fb3-4f51-a1bb-8ed05d5d63ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4b444984a704a4485618f3a93ce95a0)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:41.870463 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34e0b06-1fb3-4f51-a1bb-8ed05d5d63ed admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:41.915747 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-f34e0b06-1fb3-4f51-a1bb-8ed05d5d63ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4b444984a704a4485618f3a93ce95a0', 'request_id': 'req-f34e0b06-1fb3-4f51-a1bb-8ed05d5d63ed'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:41.921194 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-f34e0b06-1fb3-4f51-a1bb-8ed05d5d63ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4b444984a704a4485618f3a93ce95a0', 'request_id': 'req-f34e0b06-1fb3-4f51-a1bb-8ed05d5d63ed'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:41.927591 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-f34e0b06-1fb3-4f51-a1bb-8ed05d5d63ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4b444984a704a4485618f3a93ce95a0', 'request_id': 'req-f34e0b06-1fb3-4f51-a1bb-8ed05d5d63ed'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:41.941346 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-f34e0b06-1fb3-4f51-a1bb-8ed05d5d63ed admin admin] The token cache is being invalidate because project d4b444984a704a4485618f3a93ce95a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81591) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:19:41.941629 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-f34e0b06-1fb3-4f51-a1bb-8ed05d5d63ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:41.942241 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-f34e0b06-1fb3-4f51-a1bb-8ed05d5d63ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:41.943911 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1320/2620] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 9 12:19:41 2022] DELETE /identity/v3/projects/d4b444984a704a4485618f3a93ce95a0 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:19:41.948811 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898d604c-6469-447d-873f-ae84a7039e38 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:41.949266 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898d604c-6469-447d-873f-ae84a7039e38 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:41.949384 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898d604c-6469-447d-873f-ae84a7039e38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:41.968850 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-898d604c-6469-447d-873f-ae84a7039e38 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:19:41.970056 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-898d604c-6469-447d-873f-ae84a7039e38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:19:42.029185 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1301/2621] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:19:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:19:42.037884 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e1c088-8d18-4e43-a5db-80b851f71c80 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:42.038449 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e1c088-8d18-4e43-a5db-80b851f71c80 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:42.038758 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e1c088-8d18-4e43-a5db-80b851f71c80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:42.059640 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-b0e1c088-8d18-4e43-a5db-80b851f71c80 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:19:42.060839 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-b0e1c088-8d18-4e43-a5db-80b851f71c80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:19:42.069017 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1321/2622] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:19:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:19:42.073280 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898d604c-6469-447d-873f-ae84a7039e38 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:42.083073 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898d604c-6469-447d-873f-ae84a7039e38 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:42.087796 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4896671c-8a0f-4454-8d45-5cff8b0643e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:42.088396 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4896671c-8a0f-4454-8d45-5cff8b0643e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:42.088677 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4896671c-8a0f-4454-8d45-5cff8b0643e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:42.088947 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4896671c-8a0f-4454-8d45-5cff8b0643e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:42.089583 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4896671c-8a0f-4454-8d45-5cff8b0643e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:19:42.089884 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4896671c-8a0f-4454-8d45-5cff8b0643e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:42.091510 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4896671c-8a0f-4454-8d45-5cff8b0643e9 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:42.103567 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1302/2623] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 9 12:19:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:19:42.107809 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e1c088-8d18-4e43-a5db-80b851f71c80 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:42.117937 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e1c088-8d18-4e43-a5db-80b851f71c80 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:42.122491 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6756920f-4f15-4e37-914b-16672e559f49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:42.123128 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6756920f-4f15-4e37-914b-16672e559f49 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:42.123402 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6756920f-4f15-4e37-914b-16672e559f49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:42.123669 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6756920f-4f15-4e37-914b-16672e559f49 admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:42.124295 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6756920f-4f15-4e37-914b-16672e559f49 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:42.126207 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6756920f-4f15-4e37-914b-16672e559f49 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:42.137269 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:19:42.155226 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1322/2624] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 9 12:19:42 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:19:42.159399 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4896671c-8a0f-4454-8d45-5cff8b0643e9 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:42.170070 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4896671c-8a0f-4454-8d45-5cff8b0643e9 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:42.202553 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:19:42.227805 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2dd100-10b9-48e3-a1be-7b830cd44e86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:42.228411 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2dd100-10b9-48e3-a1be-7b830cd44e86 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:42.228693 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2dd100-10b9-48e3-a1be-7b830cd44e86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:42.228965 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2dd100-10b9-48e3-a1be-7b830cd44e86 admin admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:42.229597 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2dd100-10b9-48e3-a1be-7b830cd44e86 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:42.231611 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2dd100-10b9-48e3-a1be-7b830cd44e86 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:42.251999 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1303/2625] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:19:42 2022] POST /identity/v3/users => generated 292 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:19:42.256369 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6756920f-4f15-4e37-914b-16672e559f49 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:42.266482 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6756920f-4f15-4e37-914b-16672e559f49 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:42.271753 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d33c68f-76ed-43cf-a54f-bac4f175dcd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:42.272362 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d33c68f-76ed-43cf-a54f-bac4f175dcd8 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:42.272641 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d33c68f-76ed-43cf-a54f-bac4f175dcd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:42.272909 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d33c68f-76ed-43cf-a54f-bac4f175dcd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:42.273543 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d33c68f-76ed-43cf-a54f-bac4f175dcd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:42.275144 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d33c68f-76ed-43cf-a54f-bac4f175dcd8 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:42.285006 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1323/2626] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:19:42 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:19:42.289524 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2dd100-10b9-48e3-a1be-7b830cd44e86 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:42.299556 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2dd100-10b9-48e3-a1be-7b830cd44e86 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:42.304624 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e1377b-bf93-4a30-8948-eed87b685e6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:42.305247 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e1377b-bf93-4a30-8948-eed87b685e6c admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:42.305548 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e1377b-bf93-4a30-8948-eed87b685e6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:42.305821 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e1377b-bf93-4a30-8948-eed87b685e6c admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:42.306459 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e1377b-bf93-4a30-8948-eed87b685e6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:42.308097 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e1377b-bf93-4a30-8948-eed87b685e6c admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:42.318695 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1304/2627] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:19:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:19:42.322809 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d33c68f-76ed-43cf-a54f-bac4f175dcd8 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:42.332576 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d33c68f-76ed-43cf-a54f-bac4f175dcd8 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:42.337762 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330ee530-7344-4875-a0ce-6eb8a284850b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:42.338405 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330ee530-7344-4875-a0ce-6eb8a284850b admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:42.338685 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330ee530-7344-4875-a0ce-6eb8a284850b admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:42.338993 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330ee530-7344-4875-a0ce-6eb8a284850b admin admin] PATH_INFO: `/v3/projects/4287c36b293f4f97b80a41d7e9cff7ae/users/9bba82b02226432dbf771192fdd09bc3/roles/eecd2f160dd844f5906f3d345780c501` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:42.353816 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330ee530-7344-4875-a0ce-6eb8a284850b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4287c36b293f4f97b80a41d7e9cff7ae, user_id=9bba82b02226432dbf771192fdd09bc3, role_id=eecd2f160dd844f5906f3d345780c501)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:42.356507 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330ee530-7344-4875-a0ce-6eb8a284850b admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:42.369085 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1324/2628] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:19:42 2022] PUT /identity/v3/projects/4287c36b293f4f97b80a41d7e9cff7ae/users/9bba82b02226432dbf771192fdd09bc3/roles/eecd2f160dd844f5906f3d345780c501 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:19:42.380713 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7d8b14-2952-4529-8126-5dc6c0719b19 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:42.381180 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7d8b14-2952-4529-8126-5dc6c0719b19 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:42.381463 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7d8b14-2952-4529-8126-5dc6c0719b19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:42.409581 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-6d7d8b14-2952-4529-8126-5dc6c0719b19 None None] MFA Rules not processed for user `9bba82b02226432dbf771192fdd09bc3`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:19:42.410804 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-6d7d8b14-2952-4529-8126-5dc6c0719b19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:19:42.473826 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1305/2629] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:19:42 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:19:42.479050 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330ee530-7344-4875-a0ce-6eb8a284850b admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:42.540410 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330ee530-7344-4875-a0ce-6eb8a284850b admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:42.545110 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0e8eea-6ac6-4b84-be17-90c2e8490446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:42.545707 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0e8eea-6ac6-4b84-be17-90c2e8490446 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:42.545987 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0e8eea-6ac6-4b84-be17-90c2e8490446 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:42.546261 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0e8eea-6ac6-4b84-be17-90c2e8490446 admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:42.546927 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0e8eea-6ac6-4b84-be17-90c2e8490446 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:42.548839 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0e8eea-6ac6-4b84-be17-90c2e8490446 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:42.575651 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1325/2630] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 9 12:19:42 2022] POST /identity/v3/projects => generated 383 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:19:42.579801 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7d8b14-2952-4529-8126-5dc6c0719b19 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:42.591223 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7d8b14-2952-4529-8126-5dc6c0719b19 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:42.649003 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d961a8-59f0-49ea-9bab-36ab2ecd7c4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:42.649619 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d961a8-59f0-49ea-9bab-36ab2ecd7c4d admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:42.649896 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d961a8-59f0-49ea-9bab-36ab2ecd7c4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:42.650164 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d961a8-59f0-49ea-9bab-36ab2ecd7c4d admin admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:42.650807 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d961a8-59f0-49ea-9bab-36ab2ecd7c4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:42.652744 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d961a8-59f0-49ea-9bab-36ab2ecd7c4d admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:42.674004 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1306/2631] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:19:42 2022] POST /identity/v3/users => generated 298 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:19:42.678362 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0e8eea-6ac6-4b84-be17-90c2e8490446 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:42.688809 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0e8eea-6ac6-4b84-be17-90c2e8490446 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:42.693887 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2c6b97-f079-4485-9ef0-28ddab20e7ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:42.694475 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2c6b97-f079-4485-9ef0-28ddab20e7ff admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:42.694751 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2c6b97-f079-4485-9ef0-28ddab20e7ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:42.695063 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2c6b97-f079-4485-9ef0-28ddab20e7ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:42.695729 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2c6b97-f079-4485-9ef0-28ddab20e7ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:42.697758 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2c6b97-f079-4485-9ef0-28ddab20e7ff admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:42.708179 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1326/2632] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:19:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:19:42.713040 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d961a8-59f0-49ea-9bab-36ab2ecd7c4d admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:42.722799 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d961a8-59f0-49ea-9bab-36ab2ecd7c4d admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:42.728100 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8063df82-fdca-49d0-b448-f58db08b88ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:42.728736 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8063df82-fdca-49d0-b448-f58db08b88ed admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:42.729009 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8063df82-fdca-49d0-b448-f58db08b88ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:42.729276 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8063df82-fdca-49d0-b448-f58db08b88ed admin admin] PATH_INFO: `/v3/projects/10d6284567644a3590e248d29e421be7/users/d90e14594a8141b881b52e97e48cf11b/roles/3f2ca33902a04b6dbc2ef7d06d7289d3` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:42.744040 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8063df82-fdca-49d0-b448-f58db08b88ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=10d6284567644a3590e248d29e421be7, user_id=d90e14594a8141b881b52e97e48cf11b, role_id=3f2ca33902a04b6dbc2ef7d06d7289d3)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:42.746725 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8063df82-fdca-49d0-b448-f58db08b88ed admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:42.759966 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1307/2633] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:19:42 2022] PUT /identity/v3/projects/10d6284567644a3590e248d29e421be7/users/d90e14594a8141b881b52e97e48cf11b/roles/3f2ca33902a04b6dbc2ef7d06d7289d3 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:19:42.764044 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2c6b97-f079-4485-9ef0-28ddab20e7ff admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:42.774921 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2c6b97-f079-4485-9ef0-28ddab20e7ff admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:42.831670 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6271a21-1167-450e-be59-fc689155e5c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:42.832194 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6271a21-1167-450e-be59-fc689155e5c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:42.832475 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6271a21-1167-450e-be59-fc689155e5c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:42.832747 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6271a21-1167-450e-be59-fc689155e5c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:42.834031 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6271a21-1167-450e-be59-fc689155e5c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:42.835677 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6271a21-1167-450e-be59-fc689155e5c7 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:42.845345 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1327/2634] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:19:42 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:19:42.849650 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8063df82-fdca-49d0-b448-f58db08b88ed admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:42.859879 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8063df82-fdca-49d0-b448-f58db08b88ed admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:42.864486 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed4a5f5-a7d5-4641-88c9-c95ffd15b906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:42.865648 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed4a5f5-a7d5-4641-88c9-c95ffd15b906 admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:42.865923 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed4a5f5-a7d5-4641-88c9-c95ffd15b906 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:42.866192 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed4a5f5-a7d5-4641-88c9-c95ffd15b906 admin admin] PATH_INFO: `/v3/projects/10d6284567644a3590e248d29e421be7/users/d90e14594a8141b881b52e97e48cf11b/roles/3f2ca33902a04b6dbc2ef7d06d7289d3` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:42.868148 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed4a5f5-a7d5-4641-88c9-c95ffd15b906 admin admin] RBAC: Authorizing `identity:create_grant(project_id=10d6284567644a3590e248d29e421be7, user_id=d90e14594a8141b881b52e97e48cf11b, role_id=3f2ca33902a04b6dbc2ef7d06d7289d3)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:42.870763 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed4a5f5-a7d5-4641-88c9-c95ffd15b906 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:42.876261 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:19:42.882604 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1308/2635] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:19:42 2022] PUT /identity/v3/projects/10d6284567644a3590e248d29e421be7/users/d90e14594a8141b881b52e97e48cf11b/roles/3f2ca33902a04b6dbc2ef7d06d7289d3 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:19:42.892846 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6febcd89-fb56-4dbe-9e6d-27308c71d1a1 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:42.893349 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6febcd89-fb56-4dbe-9e6d-27308c71d1a1 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:42.893451 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6febcd89-fb56-4dbe-9e6d-27308c71d1a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:42.920765 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-6febcd89-fb56-4dbe-9e6d-27308c71d1a1 None None] MFA Rules not processed for user `d90e14594a8141b881b52e97e48cf11b`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:19:42.922052 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-6febcd89-fb56-4dbe-9e6d-27308c71d1a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:19:42.935317 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:19:42.990443 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1328/2636] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:19:42 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:19:43.000982 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed4a5f5-a7d5-4641-88c9-c95ffd15b906 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:43.002704 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-5ed4a5f5-a7d5-4641-88c9-c95ffd15b906 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:19:43.059696 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed4a5f5-a7d5-4641-88c9-c95ffd15b906 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:43.063344 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1c195f-258a-4b8a-969a-d243e1cdcea0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535c258100df40b58a5ee44b2c21d621', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:43.063941 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1c195f-258a-4b8a-969a-d243e1cdcea0 service cinder] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:43.064219 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1c195f-258a-4b8a-969a-d243e1cdcea0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:43.064488 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1c195f-258a-4b8a-969a-d243e1cdcea0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:43.120374 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1c195f-258a-4b8a-969a-d243e1cdcea0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:43.121731 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1c195f-258a-4b8a-969a-d243e1cdcea0 service cinder] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:43.131399 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1309/2637] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:19:42 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:19:47.347805 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6febcd89-fb56-4dbe-9e6d-27308c71d1a1 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:47.349370 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-6febcd89-fb56-4dbe-9e6d-27308c71d1a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:19:47.361123 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6febcd89-fb56-4dbe-9e6d-27308c71d1a1 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:47.365428 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62001689-6db9-400e-98af-ecdd0bb7dcd5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535c258100df40b58a5ee44b2c21d621', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:47.366022 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62001689-6db9-400e-98af-ecdd0bb7dcd5 service cinder] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:47.366301 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62001689-6db9-400e-98af-ecdd0bb7dcd5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:47.366572 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62001689-6db9-400e-98af-ecdd0bb7dcd5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:47.378078 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62001689-6db9-400e-98af-ecdd0bb7dcd5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:47.381015 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62001689-6db9-400e-98af-ecdd0bb7dcd5 service cinder] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:47.390388 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1329/2638] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:19:47 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:19:52.271095 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1c195f-258a-4b8a-969a-d243e1cdcea0 service cinder] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:52.332483 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1c195f-258a-4b8a-969a-d243e1cdcea0 service cinder] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:52.336897 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5005c0-5831-4d84-9739-86816671dc65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:52.337521 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5005c0-5831-4d84-9739-86816671dc65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:52.337887 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5005c0-5831-4d84-9739-86816671dc65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:52.338160 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5005c0-5831-4d84-9739-86816671dc65 admin admin] PATH_INFO: `/v3/users/9bba82b02226432dbf771192fdd09bc3` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:52.339295 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5005c0-5831-4d84-9739-86816671dc65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9bba82b02226432dbf771192fdd09bc3)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:52.341194 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5005c0-5831-4d84-9739-86816671dc65 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:52.388486 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-ec5005c0-5831-4d84-9739-86816671dc65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9bba82b02226432dbf771192fdd09bc3'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:52.393666 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:19:52.402000 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-ec5005c0-5831-4d84-9739-86816671dc65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9bba82b02226432dbf771192fdd09bc3'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:52.409150 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-ec5005c0-5831-4d84-9739-86816671dc65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9bba82b02226432dbf771192fdd09bc3'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:52.416284 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-ec5005c0-5831-4d84-9739-86816671dc65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9bba82b02226432dbf771192fdd09bc3'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:52.417004 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-ec5005c0-5831-4d84-9739-86816671dc65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9bba82b02226432dbf771192fdd09bc3'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:52.423111 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1310/2639] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 9 12:19:52 2022] DELETE /identity/v3/users/9bba82b02226432dbf771192fdd09bc3 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:19:52.428709 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5076569b-9e6d-4978-9c3d-4abd77b2dc43 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:52.429195 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5076569b-9e6d-4978-9c3d-4abd77b2dc43 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:52.429315 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5076569b-9e6d-4978-9c3d-4abd77b2dc43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:52.447847 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-5076569b-9e6d-4978-9c3d-4abd77b2dc43 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:19:52.449057 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-5076569b-9e6d-4978-9c3d-4abd77b2dc43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:19:52.509246 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1330/2640] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:19:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:19:52.519030 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5005c0-5831-4d84-9739-86816671dc65 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:52.520450 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-ec5005c0-5831-4d84-9739-86816671dc65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:19:52.580683 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5005c0-5831-4d84-9739-86816671dc65 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:52.584637 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7e0b01-e5fb-4ff6-ac44-563888c4e5d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:52.585209 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7e0b01-e5fb-4ff6-ac44-563888c4e5d1 service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:52.585487 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7e0b01-e5fb-4ff6-ac44-563888c4e5d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:52.585758 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7e0b01-e5fb-4ff6-ac44-563888c4e5d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:52.594736 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7e0b01-e5fb-4ff6-ac44-563888c4e5d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:52.596230 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7e0b01-e5fb-4ff6-ac44-563888c4e5d1 service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:52.606229 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1311/2641] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:19:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:19:52.874206 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5076569b-9e6d-4978-9c3d-4abd77b2dc43 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:52.875681 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-5076569b-9e6d-4978-9c3d-4abd77b2dc43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:19:52.889445 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5076569b-9e6d-4978-9c3d-4abd77b2dc43 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:52.894056 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7468cf6-ec33-4084-a2de-00ce30b05519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:52.894687 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7468cf6-ec33-4084-a2de-00ce30b05519 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:52.895611 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7468cf6-ec33-4084-a2de-00ce30b05519 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:52.895906 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7468cf6-ec33-4084-a2de-00ce30b05519 admin admin] PATH_INFO: `/v3/projects/4287c36b293f4f97b80a41d7e9cff7ae` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:52.897154 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7468cf6-ec33-4084-a2de-00ce30b05519 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4287c36b293f4f97b80a41d7e9cff7ae)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:52.898992 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7468cf6-ec33-4084-a2de-00ce30b05519 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:52.945911 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-c7468cf6-ec33-4084-a2de-00ce30b05519 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4287c36b293f4f97b80a41d7e9cff7ae', 'request_id': 'req-c7468cf6-ec33-4084-a2de-00ce30b05519'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:52.951347 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-c7468cf6-ec33-4084-a2de-00ce30b05519 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4287c36b293f4f97b80a41d7e9cff7ae', 'request_id': 'req-c7468cf6-ec33-4084-a2de-00ce30b05519'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:52.957325 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-c7468cf6-ec33-4084-a2de-00ce30b05519 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4287c36b293f4f97b80a41d7e9cff7ae', 'request_id': 'req-c7468cf6-ec33-4084-a2de-00ce30b05519'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:52.970283 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-c7468cf6-ec33-4084-a2de-00ce30b05519 admin admin] The token cache is being invalidate because project 4287c36b293f4f97b80a41d7e9cff7ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81591) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:19:52.970568 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-c7468cf6-ec33-4084-a2de-00ce30b05519 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:52.971255 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-c7468cf6-ec33-4084-a2de-00ce30b05519 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:52.972748 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1331/2642] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 9 12:19:52 2022] DELETE /identity/v3/projects/4287c36b293f4f97b80a41d7e9cff7ae => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:19:52.977609 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7e0b01-e5fb-4ff6-ac44-563888c4e5d1 service neutron] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:52.987188 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7e0b01-e5fb-4ff6-ac44-563888c4e5d1 service neutron] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:53.043048 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf67871-1960-46b5-9563-7eb333a32f9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:53.043679 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf67871-1960-46b5-9563-7eb333a32f9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:53.043952 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf67871-1960-46b5-9563-7eb333a32f9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:53.044219 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf67871-1960-46b5-9563-7eb333a32f9f admin admin] PATH_INFO: `/v3/users/d90e14594a8141b881b52e97e48cf11b` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:53.045490 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf67871-1960-46b5-9563-7eb333a32f9f admin admin] RBAC: Authorizing `identity:delete_user(user_id=d90e14594a8141b881b52e97e48cf11b)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:53.047510 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf67871-1960-46b5-9563-7eb333a32f9f admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:53.093806 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-dbf67871-1960-46b5-9563-7eb333a32f9f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd90e14594a8141b881b52e97e48cf11b'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:53.105576 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-dbf67871-1960-46b5-9563-7eb333a32f9f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd90e14594a8141b881b52e97e48cf11b'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:53.112810 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-dbf67871-1960-46b5-9563-7eb333a32f9f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd90e14594a8141b881b52e97e48cf11b'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:53.115602 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:19:53.121240 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-dbf67871-1960-46b5-9563-7eb333a32f9f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd90e14594a8141b881b52e97e48cf11b'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:53.122171 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-dbf67871-1960-46b5-9563-7eb333a32f9f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd90e14594a8141b881b52e97e48cf11b'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:53.129342 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1312/2643] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 9 12:19:52 2022] DELETE /identity/v3/users/d90e14594a8141b881b52e97e48cf11b => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:19:53.397871 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7468cf6-ec33-4084-a2de-00ce30b05519 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:53.399463 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-c7468cf6-ec33-4084-a2de-00ce30b05519 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:19:53.415691 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7468cf6-ec33-4084-a2de-00ce30b05519 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:53.436649 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:19:53.478248 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47c1979-083c-4077-be68-67915aa2812e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:53.478875 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47c1979-083c-4077-be68-67915aa2812e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:53.479177 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47c1979-083c-4077-be68-67915aa2812e admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:53.479445 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47c1979-083c-4077-be68-67915aa2812e admin admin] PATH_INFO: `/v3/projects/10d6284567644a3590e248d29e421be7` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:53.480547 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47c1979-083c-4077-be68-67915aa2812e admin admin] RBAC: Authorizing `identity:delete_project(project_id=10d6284567644a3590e248d29e421be7)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:53.482441 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47c1979-083c-4077-be68-67915aa2812e admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:53.528137 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-f47c1979-083c-4077-be68-67915aa2812e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10d6284567644a3590e248d29e421be7', 'request_id': 'req-f47c1979-083c-4077-be68-67915aa2812e'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:53.533505 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-f47c1979-083c-4077-be68-67915aa2812e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10d6284567644a3590e248d29e421be7', 'request_id': 'req-f47c1979-083c-4077-be68-67915aa2812e'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:53.540117 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-f47c1979-083c-4077-be68-67915aa2812e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10d6284567644a3590e248d29e421be7', 'request_id': 'req-f47c1979-083c-4077-be68-67915aa2812e'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:53.552360 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-f47c1979-083c-4077-be68-67915aa2812e admin admin] The token cache is being invalidate because project 10d6284567644a3590e248d29e421be7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81591) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:19:53.552695 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-f47c1979-083c-4077-be68-67915aa2812e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:53.553322 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-f47c1979-083c-4077-be68-67915aa2812e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:53.554798 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1332/2644] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 9 12:19:53 2022] DELETE /identity/v3/projects/10d6284567644a3590e248d29e421be7 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:19:53.560751 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83440958-d84f-473a-8cb3-a62a39fa403f None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:53.561225 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83440958-d84f-473a-8cb3-a62a39fa403f None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:53.561342 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83440958-d84f-473a-8cb3-a62a39fa403f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:53.580469 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-83440958-d84f-473a-8cb3-a62a39fa403f None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:19:53.581664 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-83440958-d84f-473a-8cb3-a62a39fa403f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:19:53.646561 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1313/2645] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:19:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:19:53.654972 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0bf057-e139-4b25-8759-358e14018158 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:53.655286 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0bf057-e139-4b25-8759-358e14018158 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:53.655557 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0bf057-e139-4b25-8759-358e14018158 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:53.676763 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-6e0bf057-e139-4b25-8759-358e14018158 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:19:53.677959 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-6e0bf057-e139-4b25-8759-358e14018158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:19:53.685793 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1333/2646] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:19:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:19:53.689923 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83440958-d84f-473a-8cb3-a62a39fa403f None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:53.699811 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83440958-d84f-473a-8cb3-a62a39fa403f None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:53.704332 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f3e288-d66f-4777-bd24-4ee8306a7bb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:53.704924 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f3e288-d66f-4777-bd24-4ee8306a7bb2 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:53.705200 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f3e288-d66f-4777-bd24-4ee8306a7bb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:53.705494 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f3e288-d66f-4777-bd24-4ee8306a7bb2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:53.706132 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f3e288-d66f-4777-bd24-4ee8306a7bb2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:19:53.706428 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f3e288-d66f-4777-bd24-4ee8306a7bb2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:53.708904 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f3e288-d66f-4777-bd24-4ee8306a7bb2 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:53.720285 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1314/2647] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 9 12:19:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:19:53.725333 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0bf057-e139-4b25-8759-358e14018158 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:53.734258 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0bf057-e139-4b25-8759-358e14018158 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:53.738775 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10446a0c-5e6c-4066-95e2-830a5711a447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:53.739678 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10446a0c-5e6c-4066-95e2-830a5711a447 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:53.739955 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10446a0c-5e6c-4066-95e2-830a5711a447 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:53.740220 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10446a0c-5e6c-4066-95e2-830a5711a447 admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:53.740830 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10446a0c-5e6c-4066-95e2-830a5711a447 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:53.742697 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10446a0c-5e6c-4066-95e2-830a5711a447 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:53.768992 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1334/2648] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 9 12:19:53 2022] POST /identity/v3/projects => generated 383 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:19:53.773676 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f3e288-d66f-4777-bd24-4ee8306a7bb2 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:53.783989 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f3e288-d66f-4777-bd24-4ee8306a7bb2 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:53.842427 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe210ef4-2f50-44c7-b92c-9eb291164e49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:53.843071 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe210ef4-2f50-44c7-b92c-9eb291164e49 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:53.843351 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe210ef4-2f50-44c7-b92c-9eb291164e49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:53.843622 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe210ef4-2f50-44c7-b92c-9eb291164e49 admin admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:53.844251 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe210ef4-2f50-44c7-b92c-9eb291164e49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:53.846172 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe210ef4-2f50-44c7-b92c-9eb291164e49 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:53.867037 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1315/2649] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:19:53 2022] POST /identity/v3/users => generated 292 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:19:53.871349 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10446a0c-5e6c-4066-95e2-830a5711a447 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:53.882027 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10446a0c-5e6c-4066-95e2-830a5711a447 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:53.886923 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333dbf23-e10a-48a7-ba45-63e45e1ec4c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:53.887538 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333dbf23-e10a-48a7-ba45-63e45e1ec4c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:53.887818 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333dbf23-e10a-48a7-ba45-63e45e1ec4c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:53.888093 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333dbf23-e10a-48a7-ba45-63e45e1ec4c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:53.888732 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333dbf23-e10a-48a7-ba45-63e45e1ec4c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:53.890320 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333dbf23-e10a-48a7-ba45-63e45e1ec4c8 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:53.901309 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1335/2650] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:19:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:19:53.905069 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe210ef4-2f50-44c7-b92c-9eb291164e49 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:53.915153 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe210ef4-2f50-44c7-b92c-9eb291164e49 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:53.920001 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1e71a3-41ea-49d0-ad8a-ef230a647600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:53.920594 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1e71a3-41ea-49d0-ad8a-ef230a647600 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:53.920869 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1e71a3-41ea-49d0-ad8a-ef230a647600 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:53.921134 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1e71a3-41ea-49d0-ad8a-ef230a647600 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:53.921769 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1e71a3-41ea-49d0-ad8a-ef230a647600 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:53.923392 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1e71a3-41ea-49d0-ad8a-ef230a647600 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:53.935911 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1316/2651] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:19:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:19:53.940073 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333dbf23-e10a-48a7-ba45-63e45e1ec4c8 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:53.950587 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333dbf23-e10a-48a7-ba45-63e45e1ec4c8 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:53.955182 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7656189-32eb-4644-bcd2-52d2967b33e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:53.955826 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7656189-32eb-4644-bcd2-52d2967b33e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:53.956102 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7656189-32eb-4644-bcd2-52d2967b33e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:53.956374 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7656189-32eb-4644-bcd2-52d2967b33e3 admin admin] PATH_INFO: `/v3/projects/49fb67b03f2d421fbc904d6985c68934/users/f31f6003a28640018a360a6c2e3a7fa5/roles/eecd2f160dd844f5906f3d345780c501` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:53.971038 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7656189-32eb-4644-bcd2-52d2967b33e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49fb67b03f2d421fbc904d6985c68934, user_id=f31f6003a28640018a360a6c2e3a7fa5, role_id=eecd2f160dd844f5906f3d345780c501)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:53.973677 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7656189-32eb-4644-bcd2-52d2967b33e3 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:53.986338 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1336/2652] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:19:53 2022] PUT /identity/v3/projects/49fb67b03f2d421fbc904d6985c68934/users/f31f6003a28640018a360a6c2e3a7fa5/roles/eecd2f160dd844f5906f3d345780c501 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:19:53.998649 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbefb30-d78d-459a-bb3c-3ffa21486ee2 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:53.998961 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbefb30-d78d-459a-bb3c-3ffa21486ee2 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:53.999237 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbefb30-d78d-459a-bb3c-3ffa21486ee2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:54.028428 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-abbefb30-d78d-459a-bb3c-3ffa21486ee2 None None] MFA Rules not processed for user `f31f6003a28640018a360a6c2e3a7fa5`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:19:54.029646 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-abbefb30-d78d-459a-bb3c-3ffa21486ee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:19:54.076034 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:19:54.095628 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1317/2653] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:19:53 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:19:54.107395 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7656189-32eb-4644-bcd2-52d2967b33e3 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:54.109074 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-b7656189-32eb-4644-bcd2-52d2967b33e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:19:54.162349 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:19:54.166369 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7656189-32eb-4644-bcd2-52d2967b33e3 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:54.171114 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8061073e-49f5-4206-9e00-c5687b90b4da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535c258100df40b58a5ee44b2c21d621', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:54.171932 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8061073e-49f5-4206-9e00-c5687b90b4da service cinder] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:54.172234 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8061073e-49f5-4206-9e00-c5687b90b4da service cinder] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:54.172682 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8061073e-49f5-4206-9e00-c5687b90b4da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:54.182096 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8061073e-49f5-4206-9e00-c5687b90b4da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:54.183570 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8061073e-49f5-4206-9e00-c5687b90b4da service cinder] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:54.193155 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1337/2654] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:19:54 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:19:54.212895 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbefb30-d78d-459a-bb3c-3ffa21486ee2 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:54.274943 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbefb30-d78d-459a-bb3c-3ffa21486ee2 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:54.279469 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4fe9fb-970f-45d0-8407-63242789c216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:54.280119 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4fe9fb-970f-45d0-8407-63242789c216 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:54.280394 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4fe9fb-970f-45d0-8407-63242789c216 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:54.280663 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4fe9fb-970f-45d0-8407-63242789c216 admin admin] PATH_INFO: `/v3/users/f31f6003a28640018a360a6c2e3a7fa5` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:54.281787 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4fe9fb-970f-45d0-8407-63242789c216 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f31f6003a28640018a360a6c2e3a7fa5)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:54.283732 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4fe9fb-970f-45d0-8407-63242789c216 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:54.328991 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-8d4fe9fb-970f-45d0-8407-63242789c216 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f31f6003a28640018a360a6c2e3a7fa5'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:54.340997 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-8d4fe9fb-970f-45d0-8407-63242789c216 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f31f6003a28640018a360a6c2e3a7fa5'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:54.347832 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-8d4fe9fb-970f-45d0-8407-63242789c216 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f31f6003a28640018a360a6c2e3a7fa5'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:54.355238 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-8d4fe9fb-970f-45d0-8407-63242789c216 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f31f6003a28640018a360a6c2e3a7fa5'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:54.355915 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-8d4fe9fb-970f-45d0-8407-63242789c216 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f31f6003a28640018a360a6c2e3a7fa5'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:54.361804 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1318/2655] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 9 12:19:54 2022] DELETE /identity/v3/users/f31f6003a28640018a360a6c2e3a7fa5 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:19:54.366806 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dae7ab-a0c2-4039-a1fb-f7c263a81d65 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:54.367125 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dae7ab-a0c2-4039-a1fb-f7c263a81d65 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:54.367401 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dae7ab-a0c2-4039-a1fb-f7c263a81d65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:54.386717 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-56dae7ab-a0c2-4039-a1fb-f7c263a81d65 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:19:54.387956 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-56dae7ab-a0c2-4039-a1fb-f7c263a81d65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:19:54.452782 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1338/2656] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:19:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:19:54.462385 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4fe9fb-970f-45d0-8407-63242789c216 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:54.463816 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-8d4fe9fb-970f-45d0-8407-63242789c216 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:19:54.519568 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4fe9fb-970f-45d0-8407-63242789c216 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:54.523190 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97411e9-a845-401a-829f-b69d70ea07ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:54.523782 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97411e9-a845-401a-829f-b69d70ea07ca service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:54.524054 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97411e9-a845-401a-829f-b69d70ea07ca service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:54.524321 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97411e9-a845-401a-829f-b69d70ea07ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:54.533143 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97411e9-a845-401a-829f-b69d70ea07ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:54.534553 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97411e9-a845-401a-829f-b69d70ea07ca service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:54.544349 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1319/2657] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:19:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:19:54.821600 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dae7ab-a0c2-4039-a1fb-f7c263a81d65 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:54.823103 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-56dae7ab-a0c2-4039-a1fb-f7c263a81d65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:19:54.835755 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dae7ab-a0c2-4039-a1fb-f7c263a81d65 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:54.840375 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae10d15-7916-4e79-be3a-6bf7f1d23a1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:54.841089 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae10d15-7916-4e79-be3a-6bf7f1d23a1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:54.841366 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae10d15-7916-4e79-be3a-6bf7f1d23a1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:54.841637 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae10d15-7916-4e79-be3a-6bf7f1d23a1e admin admin] PATH_INFO: `/v3/projects/49fb67b03f2d421fbc904d6985c68934` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:54.842757 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae10d15-7916-4e79-be3a-6bf7f1d23a1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=49fb67b03f2d421fbc904d6985c68934)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:54.844729 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae10d15-7916-4e79-be3a-6bf7f1d23a1e admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:54.891796 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-aae10d15-7916-4e79-be3a-6bf7f1d23a1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49fb67b03f2d421fbc904d6985c68934', 'request_id': 'req-aae10d15-7916-4e79-be3a-6bf7f1d23a1e'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:54.897951 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-aae10d15-7916-4e79-be3a-6bf7f1d23a1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49fb67b03f2d421fbc904d6985c68934', 'request_id': 'req-aae10d15-7916-4e79-be3a-6bf7f1d23a1e'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:54.904313 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-aae10d15-7916-4e79-be3a-6bf7f1d23a1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49fb67b03f2d421fbc904d6985c68934', 'request_id': 'req-aae10d15-7916-4e79-be3a-6bf7f1d23a1e'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:54.916871 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-aae10d15-7916-4e79-be3a-6bf7f1d23a1e admin admin] The token cache is being invalidate because project 49fb67b03f2d421fbc904d6985c68934 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81591) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:19:54.917213 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-aae10d15-7916-4e79-be3a-6bf7f1d23a1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:54.917829 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-aae10d15-7916-4e79-be3a-6bf7f1d23a1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:19:54.919359 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1339/2658] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 9 12:19:54 2022] DELETE /identity/v3/projects/49fb67b03f2d421fbc904d6985c68934 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:19:54.924367 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540944b0-b328-4150-af00-28f69719bf5b None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:54.924863 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540944b0-b328-4150-af00-28f69719bf5b None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:54.924972 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540944b0-b328-4150-af00-28f69719bf5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:54.945262 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-540944b0-b328-4150-af00-28f69719bf5b None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:19:54.946441 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-540944b0-b328-4150-af00-28f69719bf5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:19:54.951533 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:19:55.005889 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1320/2659] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:19:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:19:55.013634 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e9c141-3836-407a-8f3c-a60600ce4274 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:55.013918 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e9c141-3836-407a-8f3c-a60600ce4274 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:55.014189 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e9c141-3836-407a-8f3c-a60600ce4274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:55.033694 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-e3e9c141-3836-407a-8f3c-a60600ce4274 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:19:55.034930 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-e3e9c141-3836-407a-8f3c-a60600ce4274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:19:55.043260 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1340/2660] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:19:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:19:55.047287 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540944b0-b328-4150-af00-28f69719bf5b None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:55.057913 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540944b0-b328-4150-af00-28f69719bf5b None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:55.062754 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e100a2a-4820-41c0-8908-6ceda6a874d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:55.063334 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e100a2a-4820-41c0-8908-6ceda6a874d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:55.063708 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e100a2a-4820-41c0-8908-6ceda6a874d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:55.063880 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e100a2a-4820-41c0-8908-6ceda6a874d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:55.064574 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e100a2a-4820-41c0-8908-6ceda6a874d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:19:55.064815 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e100a2a-4820-41c0-8908-6ceda6a874d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:55.066406 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e100a2a-4820-41c0-8908-6ceda6a874d1 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:55.082431 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1321/2661] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 9 12:19:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:19:55.083936 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e9c141-3836-407a-8f3c-a60600ce4274 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:55.093202 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e9c141-3836-407a-8f3c-a60600ce4274 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:55.097786 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303cb355-5fcc-44de-90f2-065ce43a7ae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:55.098418 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303cb355-5fcc-44de-90f2-065ce43a7ae6 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:55.098696 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303cb355-5fcc-44de-90f2-065ce43a7ae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:55.098997 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303cb355-5fcc-44de-90f2-065ce43a7ae6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:55.099628 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303cb355-5fcc-44de-90f2-065ce43a7ae6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:55.101512 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303cb355-5fcc-44de-90f2-065ce43a7ae6 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:55.128026 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1341/2662] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 9 12:19:55 2022] POST /identity/v3/projects => generated 375 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:19:55.133173 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e100a2a-4820-41c0-8908-6ceda6a874d1 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:55.143625 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e100a2a-4820-41c0-8908-6ceda6a874d1 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:55.202270 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363d7a04-dbfc-46b4-8a54-8347da6c9ba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:55.202869 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363d7a04-dbfc-46b4-8a54-8347da6c9ba8 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:55.203176 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363d7a04-dbfc-46b4-8a54-8347da6c9ba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:55.203445 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363d7a04-dbfc-46b4-8a54-8347da6c9ba8 admin admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:55.204143 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363d7a04-dbfc-46b4-8a54-8347da6c9ba8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:55.206424 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363d7a04-dbfc-46b4-8a54-8347da6c9ba8 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:55.233986 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1322/2663] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:19:55 2022] POST /identity/v3/users => generated 288 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:19:55.235443 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303cb355-5fcc-44de-90f2-065ce43a7ae6 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:55.246379 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303cb355-5fcc-44de-90f2-065ce43a7ae6 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:55.251007 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b9d704-b57e-48a3-9c31-0b0d5846bd04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:55.251607 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b9d704-b57e-48a3-9c31-0b0d5846bd04 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:55.251885 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b9d704-b57e-48a3-9c31-0b0d5846bd04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:55.252155 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b9d704-b57e-48a3-9c31-0b0d5846bd04 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:55.252807 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b9d704-b57e-48a3-9c31-0b0d5846bd04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:55.254382 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b9d704-b57e-48a3-9c31-0b0d5846bd04 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:55.266254 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1342/2664] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:19:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 12:19:55.270345 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363d7a04-dbfc-46b4-8a54-8347da6c9ba8 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:55.280416 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363d7a04-dbfc-46b4-8a54-8347da6c9ba8 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:55.285114 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d78776d-1199-47ea-bb31-ba0212fe1ba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:55.286268 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d78776d-1199-47ea-bb31-ba0212fe1ba2 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:55.286546 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d78776d-1199-47ea-bb31-ba0212fe1ba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:55.286816 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d78776d-1199-47ea-bb31-ba0212fe1ba2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:55.287497 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d78776d-1199-47ea-bb31-ba0212fe1ba2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:55.289094 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d78776d-1199-47ea-bb31-ba0212fe1ba2 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:55.300023 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1323/2665] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:19:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:19:55.309560 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b9d704-b57e-48a3-9c31-0b0d5846bd04 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:55.321888 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b9d704-b57e-48a3-9c31-0b0d5846bd04 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:55.327112 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe015764-daed-4218-a3bf-e95e97868653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:55.327834 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe015764-daed-4218-a3bf-e95e97868653 admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:55.328112 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe015764-daed-4218-a3bf-e95e97868653 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:55.328382 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe015764-daed-4218-a3bf-e95e97868653 admin admin] PATH_INFO: `/v3/projects/a9e5a54b1a8b4ed589ccbf2d02c5f004/users/07b4a31a7126403ea4c38fe695f9186f/roles/eecd2f160dd844f5906f3d345780c501` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:55.344016 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe015764-daed-4218-a3bf-e95e97868653 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9e5a54b1a8b4ed589ccbf2d02c5f004, user_id=07b4a31a7126403ea4c38fe695f9186f, role_id=eecd2f160dd844f5906f3d345780c501)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:55.346648 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe015764-daed-4218-a3bf-e95e97868653 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:55.360305 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1343/2666] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:19:55 2022] PUT /identity/v3/projects/a9e5a54b1a8b4ed589ccbf2d02c5f004/users/07b4a31a7126403ea4c38fe695f9186f/roles/eecd2f160dd844f5906f3d345780c501 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:19:55.371443 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dc5b01-be4d-4915-978b-2df8a28b22e7 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:55.371731 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dc5b01-be4d-4915-978b-2df8a28b22e7 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:55.372003 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dc5b01-be4d-4915-978b-2df8a28b22e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:55.400126 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-79dc5b01-be4d-4915-978b-2df8a28b22e7 None None] MFA Rules not processed for user `07b4a31a7126403ea4c38fe695f9186f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:19:55.401554 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-79dc5b01-be4d-4915-978b-2df8a28b22e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:19:55.469149 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1324/2667] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:19:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:19:55.474394 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe015764-daed-4218-a3bf-e95e97868653 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:55.493988 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:19:55.545867 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe015764-daed-4218-a3bf-e95e97868653 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:55.554052 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cfda31-a7ca-4a19-b352-217999eae3a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:55.554052 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cfda31-a7ca-4a19-b352-217999eae3a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:55.554052 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cfda31-a7ca-4a19-b352-217999eae3a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:55.554052 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cfda31-a7ca-4a19-b352-217999eae3a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:55.554510 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cfda31-a7ca-4a19-b352-217999eae3a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:55.555248 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cfda31-a7ca-4a19-b352-217999eae3a8 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:55.582051 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1344/2668] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 9 12:19:55 2022] POST /identity/v3/projects => generated 375 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:19:55.586441 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dc5b01-be4d-4915-978b-2df8a28b22e7 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:55.597258 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dc5b01-be4d-4915-978b-2df8a28b22e7 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:55.601692 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:19:55.664313 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a1d64c-beae-435f-95d2-187a11a55dad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:55.664926 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a1d64c-beae-435f-95d2-187a11a55dad admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:55.665205 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a1d64c-beae-435f-95d2-187a11a55dad admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:55.665473 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a1d64c-beae-435f-95d2-187a11a55dad admin admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:55.666097 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a1d64c-beae-435f-95d2-187a11a55dad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:55.668061 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a1d64c-beae-435f-95d2-187a11a55dad admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:55.689573 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1325/2669] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:19:55 2022] POST /identity/v3/users => generated 288 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:19:55.693803 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cfda31-a7ca-4a19-b352-217999eae3a8 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:55.703875 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cfda31-a7ca-4a19-b352-217999eae3a8 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:55.709208 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a33927-d82b-4aba-83f3-5b70adb66c9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:55.709897 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a33927-d82b-4aba-83f3-5b70adb66c9d admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:55.710173 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a33927-d82b-4aba-83f3-5b70adb66c9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:55.710441 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a33927-d82b-4aba-83f3-5b70adb66c9d admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:55.711107 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a33927-d82b-4aba-83f3-5b70adb66c9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:55.712747 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a33927-d82b-4aba-83f3-5b70adb66c9d admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:55.724123 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1345/2670] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:19:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:19:55.728318 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a1d64c-beae-435f-95d2-187a11a55dad admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:55.738803 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a1d64c-beae-435f-95d2-187a11a55dad admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:55.743303 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008115a3-72ce-4f29-b54a-49710ca9cc96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:55.743920 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008115a3-72ce-4f29-b54a-49710ca9cc96 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:55.744194 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008115a3-72ce-4f29-b54a-49710ca9cc96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:55.744461 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008115a3-72ce-4f29-b54a-49710ca9cc96 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:55.745093 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008115a3-72ce-4f29-b54a-49710ca9cc96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:55.746693 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008115a3-72ce-4f29-b54a-49710ca9cc96 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:55.756870 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1326/2671] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:19:55 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 12:19:55.760735 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a33927-d82b-4aba-83f3-5b70adb66c9d admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:55.771241 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a33927-d82b-4aba-83f3-5b70adb66c9d admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:55.775836 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75983ee8-f2dc-4442-be8d-b67c86480e23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:55.776480 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75983ee8-f2dc-4442-be8d-b67c86480e23 admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:55.776760 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75983ee8-f2dc-4442-be8d-b67c86480e23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:55.777034 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75983ee8-f2dc-4442-be8d-b67c86480e23 admin admin] PATH_INFO: `/v3/projects/06fe993f5b0046eda86c107fa3bc377b/users/eff4331309904d6fb43350bf1eedbfc0/roles/eecd2f160dd844f5906f3d345780c501` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:55.793049 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75983ee8-f2dc-4442-be8d-b67c86480e23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06fe993f5b0046eda86c107fa3bc377b, user_id=eff4331309904d6fb43350bf1eedbfc0, role_id=eecd2f160dd844f5906f3d345780c501)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:55.795747 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75983ee8-f2dc-4442-be8d-b67c86480e23 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:55.808689 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1346/2672] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:19:55 2022] PUT /identity/v3/projects/06fe993f5b0046eda86c107fa3bc377b/users/eff4331309904d6fb43350bf1eedbfc0/roles/eecd2f160dd844f5906f3d345780c501 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:19:55.820481 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58a0173-850e-4bbf-be6e-4ef98d1f7a93 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:55.820770 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58a0173-850e-4bbf-be6e-4ef98d1f7a93 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:55.821047 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58a0173-850e-4bbf-be6e-4ef98d1f7a93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:55.849781 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-a58a0173-850e-4bbf-be6e-4ef98d1f7a93 None None] MFA Rules not processed for user `eff4331309904d6fb43350bf1eedbfc0`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:19:55.851018 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-a58a0173-850e-4bbf-be6e-4ef98d1f7a93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:19:55.913591 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1327/2673] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:19:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:19:55.918217 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75983ee8-f2dc-4442-be8d-b67c86480e23 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:55.978550 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75983ee8-f2dc-4442-be8d-b67c86480e23 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:55.983147 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b422a1-d3f2-4d35-9015-fdea7cfec8bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:55.983748 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b422a1-d3f2-4d35-9015-fdea7cfec8bd admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:55.984025 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b422a1-d3f2-4d35-9015-fdea7cfec8bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:55.984296 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b422a1-d3f2-4d35-9015-fdea7cfec8bd admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:55.984910 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b422a1-d3f2-4d35-9015-fdea7cfec8bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:55.986807 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b422a1-d3f2-4d35-9015-fdea7cfec8bd admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:56.013486 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1347/2674] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 9 12:19:55 2022] POST /identity/v3/projects => generated 373 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:19:56.017716 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58a0173-850e-4bbf-be6e-4ef98d1f7a93 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:56.029008 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58a0173-850e-4bbf-be6e-4ef98d1f7a93 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:56.084906 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba46d84a-eab7-498d-8af2-b8ff88335865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:56.085508 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba46d84a-eab7-498d-8af2-b8ff88335865 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:56.085784 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba46d84a-eab7-498d-8af2-b8ff88335865 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:56.086054 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba46d84a-eab7-498d-8af2-b8ff88335865 admin admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:56.086688 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba46d84a-eab7-498d-8af2-b8ff88335865 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:56.088629 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba46d84a-eab7-498d-8af2-b8ff88335865 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:56.110518 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1328/2675] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:19:56 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:19:56.114972 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b422a1-d3f2-4d35-9015-fdea7cfec8bd admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:56.125070 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b422a1-d3f2-4d35-9015-fdea7cfec8bd admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:56.130375 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14347bfa-c000-43f2-ab5e-1416ec1df098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:56.130999 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14347bfa-c000-43f2-ab5e-1416ec1df098 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:56.131295 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14347bfa-c000-43f2-ab5e-1416ec1df098 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:56.131552 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14347bfa-c000-43f2-ab5e-1416ec1df098 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:56.132191 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14347bfa-c000-43f2-ab5e-1416ec1df098 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:56.133813 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14347bfa-c000-43f2-ab5e-1416ec1df098 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:56.144399 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1348/2676] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:19:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 12:19:56.149370 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba46d84a-eab7-498d-8af2-b8ff88335865 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:56.155764 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:19:56.159933 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba46d84a-eab7-498d-8af2-b8ff88335865 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:56.165324 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e8460d-86e8-4cff-9ad4-47190dd384d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:56.165976 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e8460d-86e8-4cff-9ad4-47190dd384d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:56.166256 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e8460d-86e8-4cff-9ad4-47190dd384d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:56.166539 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e8460d-86e8-4cff-9ad4-47190dd384d9 admin admin] PATH_INFO: `/v3/projects/778e86e01aae4b738f1e13f6ec5a1bfe/users/038fc6a52d5d4295a6a82214eba25a12/roles/3f2ca33902a04b6dbc2ef7d06d7289d3` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:56.181393 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e8460d-86e8-4cff-9ad4-47190dd384d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=778e86e01aae4b738f1e13f6ec5a1bfe, user_id=038fc6a52d5d4295a6a82214eba25a12, role_id=3f2ca33902a04b6dbc2ef7d06d7289d3)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:56.184166 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e8460d-86e8-4cff-9ad4-47190dd384d9 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:56.197328 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1329/2677] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:19:56 2022] PUT /identity/v3/projects/778e86e01aae4b738f1e13f6ec5a1bfe/users/038fc6a52d5d4295a6a82214eba25a12/roles/3f2ca33902a04b6dbc2ef7d06d7289d3 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:19:56.201306 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14347bfa-c000-43f2-ab5e-1416ec1df098 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:56.214041 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14347bfa-c000-43f2-ab5e-1416ec1df098 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:56.246269 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:19:56.272468 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181ebf00-8cb3-4880-ad76-01888cca222b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:56.273006 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181ebf00-8cb3-4880-ad76-01888cca222b admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:56.273289 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181ebf00-8cb3-4880-ad76-01888cca222b admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:56.273564 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181ebf00-8cb3-4880-ad76-01888cca222b admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:56.274197 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181ebf00-8cb3-4880-ad76-01888cca222b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:56.275829 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181ebf00-8cb3-4880-ad76-01888cca222b admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:56.285383 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1349/2678] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:19:56 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:19:56.289526 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e8460d-86e8-4cff-9ad4-47190dd384d9 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:56.299905 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e8460d-86e8-4cff-9ad4-47190dd384d9 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:56.304986 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1414908d-9db9-43c4-8496-f7b8f0d2a9c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:56.305629 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1414908d-9db9-43c4-8496-f7b8f0d2a9c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:56.305903 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1414908d-9db9-43c4-8496-f7b8f0d2a9c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:56.306169 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1414908d-9db9-43c4-8496-f7b8f0d2a9c5 admin admin] PATH_INFO: `/v3/projects/778e86e01aae4b738f1e13f6ec5a1bfe/users/038fc6a52d5d4295a6a82214eba25a12/roles/3f2ca33902a04b6dbc2ef7d06d7289d3` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:56.308079 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1414908d-9db9-43c4-8496-f7b8f0d2a9c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=778e86e01aae4b738f1e13f6ec5a1bfe, user_id=038fc6a52d5d4295a6a82214eba25a12, role_id=3f2ca33902a04b6dbc2ef7d06d7289d3)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:56.310764 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1414908d-9db9-43c4-8496-f7b8f0d2a9c5 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:56.322012 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1330/2679] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:19:56 2022] PUT /identity/v3/projects/778e86e01aae4b738f1e13f6ec5a1bfe/users/038fc6a52d5d4295a6a82214eba25a12/roles/3f2ca33902a04b6dbc2ef7d06d7289d3 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:19:56.332315 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5697d3ac-c07e-47cd-8533-01604566d61b None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:56.333193 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5697d3ac-c07e-47cd-8533-01604566d61b None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:56.333470 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5697d3ac-c07e-47cd-8533-01604566d61b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:56.360476 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-5697d3ac-c07e-47cd-8533-01604566d61b None None] MFA Rules not processed for user `038fc6a52d5d4295a6a82214eba25a12`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:19:56.361688 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-5697d3ac-c07e-47cd-8533-01604566d61b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:19:56.427766 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1350/2680] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:19:56 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:19:56.439173 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1414908d-9db9-43c4-8496-f7b8f0d2a9c5 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:56.440983 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-1414908d-9db9-43c4-8496-f7b8f0d2a9c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:19:56.498196 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1414908d-9db9-43c4-8496-f7b8f0d2a9c5 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:56.502605 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d99ad3a-e754-419c-9614-ee9e3824a349 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535c258100df40b58a5ee44b2c21d621', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:56.503414 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d99ad3a-e754-419c-9614-ee9e3824a349 service cinder] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:56.503697 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d99ad3a-e754-419c-9614-ee9e3824a349 service cinder] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:56.503972 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d99ad3a-e754-419c-9614-ee9e3824a349 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:56.562187 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d99ad3a-e754-419c-9614-ee9e3824a349 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:56.563580 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d99ad3a-e754-419c-9614-ee9e3824a349 service cinder] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:56.573084 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1331/2681] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:19:56 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:19:58.152474 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5697d3ac-c07e-47cd-8533-01604566d61b None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:58.159487 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-5697d3ac-c07e-47cd-8533-01604566d61b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:19:58.172897 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5697d3ac-c07e-47cd-8533-01604566d61b None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:58.177470 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8546c985-186b-4258-9dfd-90f89e81837d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535c258100df40b58a5ee44b2c21d621', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:58.178309 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8546c985-186b-4258-9dfd-90f89e81837d service cinder] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:58.178760 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8546c985-186b-4258-9dfd-90f89e81837d service cinder] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:58.179230 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8546c985-186b-4258-9dfd-90f89e81837d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:58.246331 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8546c985-186b-4258-9dfd-90f89e81837d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:58.247945 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8546c985-186b-4258-9dfd-90f89e81837d service cinder] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:58.260293 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1351/2682] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:19:58 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:19:58.826106 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d99ad3a-e754-419c-9614-ee9e3824a349 service cinder] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:19:58.834870 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d99ad3a-e754-419c-9614-ee9e3824a349 service cinder] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:19:58.838665 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d679cc-6e21-43aa-895a-b8fdc432128a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535c258100df40b58a5ee44b2c21d621', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:19:58.839297 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d679cc-6e21-43aa-895a-b8fdc432128a service cinder] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:19:58.839574 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d679cc-6e21-43aa-895a-b8fdc432128a service cinder] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:19:58.839842 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d679cc-6e21-43aa-895a-b8fdc432128a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:19:58.849223 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d679cc-6e21-43aa-895a-b8fdc432128a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:19:58.850683 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d679cc-6e21-43aa-895a-b8fdc432128a service cinder] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:19:58.859983 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1332/2683] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:19:58 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:20:02.111149 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8546c985-186b-4258-9dfd-90f89e81837d service cinder] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:20:02.112529 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-8546c985-186b-4258-9dfd-90f89e81837d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:20:02.168750 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8546c985-186b-4258-9dfd-90f89e81837d service cinder] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:20:02.172477 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0bdf90-ff72-43d9-9b19-99541f666fbe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:20:02.173078 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0bdf90-ff72-43d9-9b19-99541f666fbe service neutron] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:20:02.173358 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0bdf90-ff72-43d9-9b19-99541f666fbe service neutron] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:20:02.173626 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0bdf90-ff72-43d9-9b19-99541f666fbe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:20:02.175378 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-1f0bdf90-ff72-43d9-9b19-99541f666fbe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:20:02.185245 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:20:02.239847 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0bdf90-ff72-43d9-9b19-99541f666fbe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:20:02.241284 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0bdf90-ff72-43d9-9b19-99541f666fbe service neutron] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:20:02.250822 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1352/2684] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:20:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:20:02.616473 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d679cc-6e21-43aa-895a-b8fdc432128a service cinder] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:20:02.617913 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-33d679cc-6e21-43aa-895a-b8fdc432128a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:20:02.634616 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d679cc-6e21-43aa-895a-b8fdc432128a service cinder] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:20:02.638215 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189bce0f-c6e7-47b6-b018-439973a4994f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:20:02.638807 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189bce0f-c6e7-47b6-b018-439973a4994f service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:20:02.639119 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189bce0f-c6e7-47b6-b018-439973a4994f service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:20:02.639385 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189bce0f-c6e7-47b6-b018-439973a4994f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:20:02.641291 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-189bce0f-c6e7-47b6-b018-439973a4994f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:20:02.715181 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189bce0f-c6e7-47b6-b018-439973a4994f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:20:02.716725 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189bce0f-c6e7-47b6-b018-439973a4994f service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:20:02.728853 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1333/2685] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:20:02 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:20:02.743709 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0bdf90-ff72-43d9-9b19-99541f666fbe service neutron] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:20:02.753729 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0bdf90-ff72-43d9-9b19-99541f666fbe service neutron] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:20:02.759208 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19b5dd6-6e70-44b6-a80c-4a9984b8abc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:20:02.759877 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19b5dd6-6e70-44b6-a80c-4a9984b8abc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:20:02.760161 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19b5dd6-6e70-44b6-a80c-4a9984b8abc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:20:02.760435 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19b5dd6-6e70-44b6-a80c-4a9984b8abc0 admin admin] PATH_INFO: `/v3/users/07b4a31a7126403ea4c38fe695f9186f` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:20:02.761607 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19b5dd6-6e70-44b6-a80c-4a9984b8abc0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=07b4a31a7126403ea4c38fe695f9186f)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:20:02.763614 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19b5dd6-6e70-44b6-a80c-4a9984b8abc0 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:20:02.826914 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-b19b5dd6-6e70-44b6-a80c-4a9984b8abc0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07b4a31a7126403ea4c38fe695f9186f'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:20:02.835865 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-b19b5dd6-6e70-44b6-a80c-4a9984b8abc0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07b4a31a7126403ea4c38fe695f9186f'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:20:02.850748 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-b19b5dd6-6e70-44b6-a80c-4a9984b8abc0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07b4a31a7126403ea4c38fe695f9186f'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:20:02.863609 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-b19b5dd6-6e70-44b6-a80c-4a9984b8abc0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07b4a31a7126403ea4c38fe695f9186f'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:20:02.864846 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-b19b5dd6-6e70-44b6-a80c-4a9984b8abc0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07b4a31a7126403ea4c38fe695f9186f'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:20:02.872416 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1353/2686] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 9 12:20:02 2022] DELETE /identity/v3/users/07b4a31a7126403ea4c38fe695f9186f => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:20:02.876962 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1191cfe8-5cf3-450a-89cb-a1c8fdcbca1d None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:20:02.877255 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1191cfe8-5cf3-450a-89cb-a1c8fdcbca1d None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:20:02.877523 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1191cfe8-5cf3-450a-89cb-a1c8fdcbca1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:20:02.892856 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:20:02.900735 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-1191cfe8-5cf3-450a-89cb-a1c8fdcbca1d None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:20:02.903140 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-1191cfe8-5cf3-450a-89cb-a1c8fdcbca1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:20:02.971210 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1334/2687] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:20:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:20:02.983004 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19b5dd6-6e70-44b6-a80c-4a9984b8abc0 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:20:02.984797 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-b19b5dd6-6e70-44b6-a80c-4a9984b8abc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:20:03.049867 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19b5dd6-6e70-44b6-a80c-4a9984b8abc0 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:20:03.055251 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4a2510-b474-4895-a42b-513f9a7ac6f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:20:03.056462 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4a2510-b474-4895-a42b-513f9a7ac6f3 service neutron] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:20:03.056895 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4a2510-b474-4895-a42b-513f9a7ac6f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:20:03.057320 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4a2510-b474-4895-a42b-513f9a7ac6f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:20:03.068770 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4a2510-b474-4895-a42b-513f9a7ac6f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:20:03.070597 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4a2510-b474-4895-a42b-513f9a7ac6f3 service neutron] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:20:03.081168 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1354/2688] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:20:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:20:03.719981 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1191cfe8-5cf3-450a-89cb-a1c8fdcbca1d None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:20:03.721938 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-1191cfe8-5cf3-450a-89cb-a1c8fdcbca1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:20:03.740977 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1191cfe8-5cf3-450a-89cb-a1c8fdcbca1d None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:20:03.762305 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed964628-ce58-46de-a095-2eb3a8b25a46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:20:03.763166 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed964628-ce58-46de-a095-2eb3a8b25a46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:20:03.763616 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed964628-ce58-46de-a095-2eb3a8b25a46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:20:03.763976 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed964628-ce58-46de-a095-2eb3a8b25a46 admin admin] PATH_INFO: `/v3/projects/a9e5a54b1a8b4ed589ccbf2d02c5f004` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:20:03.765712 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed964628-ce58-46de-a095-2eb3a8b25a46 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9e5a54b1a8b4ed589ccbf2d02c5f004)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:20:03.772466 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed964628-ce58-46de-a095-2eb3a8b25a46 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:20:03.840376 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-ed964628-ce58-46de-a095-2eb3a8b25a46 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9e5a54b1a8b4ed589ccbf2d02c5f004', 'request_id': 'req-ed964628-ce58-46de-a095-2eb3a8b25a46'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:20:03.851667 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-ed964628-ce58-46de-a095-2eb3a8b25a46 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9e5a54b1a8b4ed589ccbf2d02c5f004', 'request_id': 'req-ed964628-ce58-46de-a095-2eb3a8b25a46'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:20:03.867062 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-ed964628-ce58-46de-a095-2eb3a8b25a46 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9e5a54b1a8b4ed589ccbf2d02c5f004', 'request_id': 'req-ed964628-ce58-46de-a095-2eb3a8b25a46'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:20:03.886850 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-ed964628-ce58-46de-a095-2eb3a8b25a46 admin admin] The token cache is being invalidate because project a9e5a54b1a8b4ed589ccbf2d02c5f004 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:20:03.887270 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-ed964628-ce58-46de-a095-2eb3a8b25a46 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:20:03.888035 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-ed964628-ce58-46de-a095-2eb3a8b25a46 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:20:03.890145 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1335/2689] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 9 12:20:03 2022] DELETE /identity/v3/projects/a9e5a54b1a8b4ed589ccbf2d02c5f004 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:20:03.894066 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4a2510-b474-4895-a42b-513f9a7ac6f3 service neutron] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:20:03.914473 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4a2510-b474-4895-a42b-513f9a7ac6f3 service neutron] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:20:03.935227 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:20:03.990135 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a07033-8a8c-4cbb-91e8-27c5f089a2a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:20:03.990966 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a07033-8a8c-4cbb-91e8-27c5f089a2a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:20:03.991350 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a07033-8a8c-4cbb-91e8-27c5f089a2a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:20:03.991994 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a07033-8a8c-4cbb-91e8-27c5f089a2a3 admin admin] PATH_INFO: `/v3/users/eff4331309904d6fb43350bf1eedbfc0` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:20:03.993305 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a07033-8a8c-4cbb-91e8-27c5f089a2a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eff4331309904d6fb43350bf1eedbfc0)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:20:03.995401 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a07033-8a8c-4cbb-91e8-27c5f089a2a3 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:20:04.059460 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-38a07033-8a8c-4cbb-91e8-27c5f089a2a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eff4331309904d6fb43350bf1eedbfc0'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:20:04.068805 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-38a07033-8a8c-4cbb-91e8-27c5f089a2a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eff4331309904d6fb43350bf1eedbfc0'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:20:04.090097 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-38a07033-8a8c-4cbb-91e8-27c5f089a2a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eff4331309904d6fb43350bf1eedbfc0'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:20:04.104101 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-38a07033-8a8c-4cbb-91e8-27c5f089a2a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eff4331309904d6fb43350bf1eedbfc0'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:20:04.104658 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-38a07033-8a8c-4cbb-91e8-27c5f089a2a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eff4331309904d6fb43350bf1eedbfc0'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:20:04.119109 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1355/2690] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 9 12:20:03 2022] DELETE /identity/v3/users/eff4331309904d6fb43350bf1eedbfc0 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:20:04.419612 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed964628-ce58-46de-a095-2eb3a8b25a46 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:20:04.420486 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-ed964628-ce58-46de-a095-2eb3a8b25a46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:20:04.448525 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed964628-ce58-46de-a095-2eb3a8b25a46 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:20:04.530046 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11e5781-a9bd-4045-a7cc-30800a341fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:20:04.531403 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11e5781-a9bd-4045-a7cc-30800a341fc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:20:04.532042 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11e5781-a9bd-4045-a7cc-30800a341fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:20:04.532825 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11e5781-a9bd-4045-a7cc-30800a341fc7 admin admin] PATH_INFO: `/v3/projects/06fe993f5b0046eda86c107fa3bc377b` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:20:04.534589 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11e5781-a9bd-4045-a7cc-30800a341fc7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=06fe993f5b0046eda86c107fa3bc377b)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:20:04.537509 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11e5781-a9bd-4045-a7cc-30800a341fc7 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:20:04.604362 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:20:04.649734 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-c11e5781-a9bd-4045-a7cc-30800a341fc7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06fe993f5b0046eda86c107fa3bc377b', 'request_id': 'req-c11e5781-a9bd-4045-a7cc-30800a341fc7'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:20:04.656861 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-c11e5781-a9bd-4045-a7cc-30800a341fc7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06fe993f5b0046eda86c107fa3bc377b', 'request_id': 'req-c11e5781-a9bd-4045-a7cc-30800a341fc7'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:20:04.666428 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-c11e5781-a9bd-4045-a7cc-30800a341fc7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06fe993f5b0046eda86c107fa3bc377b', 'request_id': 'req-c11e5781-a9bd-4045-a7cc-30800a341fc7'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:20:04.682352 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-c11e5781-a9bd-4045-a7cc-30800a341fc7 admin admin] The token cache is being invalidate because project 06fe993f5b0046eda86c107fa3bc377b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:20:04.682659 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-c11e5781-a9bd-4045-a7cc-30800a341fc7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:20:04.683552 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-c11e5781-a9bd-4045-a7cc-30800a341fc7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:20:04.686081 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1336/2691] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 9 12:20:04 2022] DELETE /identity/v3/projects/06fe993f5b0046eda86c107fa3bc377b => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:20:04.689961 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a07033-8a8c-4cbb-91e8-27c5f089a2a3 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:20:04.692052 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-38a07033-8a8c-4cbb-91e8-27c5f089a2a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:20:04.705795 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a07033-8a8c-4cbb-91e8-27c5f089a2a3 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:20:04.901420 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8f9077-9bd4-4c78-989d-6934846ef692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:20:04.903640 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8f9077-9bd4-4c78-989d-6934846ef692 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:20:04.903640 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8f9077-9bd4-4c78-989d-6934846ef692 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:20:04.903640 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8f9077-9bd4-4c78-989d-6934846ef692 admin admin] PATH_INFO: `/v3/users/038fc6a52d5d4295a6a82214eba25a12` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:20:04.904217 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8f9077-9bd4-4c78-989d-6934846ef692 admin admin] RBAC: Authorizing `identity:delete_user(user_id=038fc6a52d5d4295a6a82214eba25a12)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:20:04.906443 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8f9077-9bd4-4c78-989d-6934846ef692 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:20:04.961930 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:20:04.969832 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-af8f9077-9bd4-4c78-989d-6934846ef692 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '038fc6a52d5d4295a6a82214eba25a12'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:20:04.980800 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-af8f9077-9bd4-4c78-989d-6934846ef692 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '038fc6a52d5d4295a6a82214eba25a12'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:20:04.999567 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-af8f9077-9bd4-4c78-989d-6934846ef692 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '038fc6a52d5d4295a6a82214eba25a12'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:20:05.009380 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-af8f9077-9bd4-4c78-989d-6934846ef692 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '038fc6a52d5d4295a6a82214eba25a12'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:20:05.010424 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-af8f9077-9bd4-4c78-989d-6934846ef692 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '038fc6a52d5d4295a6a82214eba25a12'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:20:05.018761 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1356/2692] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 9 12:20:04 2022] DELETE /identity/v3/users/038fc6a52d5d4295a6a82214eba25a12 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:20:05.515520 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11e5781-a9bd-4045-a7cc-30800a341fc7 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:20:05.517719 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-c11e5781-a9bd-4045-a7cc-30800a341fc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:20:05.531580 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11e5781-a9bd-4045-a7cc-30800a341fc7 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:20:05.625073 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fc87f8-607b-4d1e-a13a-1828d7e4be03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:20:05.625993 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fc87f8-607b-4d1e-a13a-1828d7e4be03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:20:05.626469 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fc87f8-607b-4d1e-a13a-1828d7e4be03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:20:05.626930 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fc87f8-607b-4d1e-a13a-1828d7e4be03 admin admin] PATH_INFO: `/v3/projects/778e86e01aae4b738f1e13f6ec5a1bfe` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:20:05.628196 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fc87f8-607b-4d1e-a13a-1828d7e4be03 admin admin] RBAC: Authorizing `identity:delete_project(project_id=778e86e01aae4b738f1e13f6ec5a1bfe)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:20:05.630810 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fc87f8-607b-4d1e-a13a-1828d7e4be03 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:20:05.686074 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-e3fc87f8-607b-4d1e-a13a-1828d7e4be03 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '778e86e01aae4b738f1e13f6ec5a1bfe', 'request_id': 'req-e3fc87f8-607b-4d1e-a13a-1828d7e4be03'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:20:05.693891 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-e3fc87f8-607b-4d1e-a13a-1828d7e4be03 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '778e86e01aae4b738f1e13f6ec5a1bfe', 'request_id': 'req-e3fc87f8-607b-4d1e-a13a-1828d7e4be03'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:20:05.704801 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-e3fc87f8-607b-4d1e-a13a-1828d7e4be03 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '778e86e01aae4b738f1e13f6ec5a1bfe', 'request_id': 'req-e3fc87f8-607b-4d1e-a13a-1828d7e4be03'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:20:05.718271 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-e3fc87f8-607b-4d1e-a13a-1828d7e4be03 admin admin] The token cache is being invalidate because project 778e86e01aae4b738f1e13f6ec5a1bfe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:20:05.718798 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-e3fc87f8-607b-4d1e-a13a-1828d7e4be03 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:20:05.719521 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-e3fc87f8-607b-4d1e-a13a-1828d7e4be03 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:20:05.721430 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1337/2693] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 9 12:20:05 2022] DELETE /identity/v3/projects/778e86e01aae4b738f1e13f6ec5a1bfe => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:20:05.726102 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b5cc78-13fc-49c4-8a69-99afa5e0a0f1 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:20:05.726558 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b5cc78-13fc-49c4-8a69-99afa5e0a0f1 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:20:05.727339 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b5cc78-13fc-49c4-8a69-99afa5e0a0f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:20:05.751798 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-67b5cc78-13fc-49c4-8a69-99afa5e0a0f1 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:20:05.753758 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-67b5cc78-13fc-49c4-8a69-99afa5e0a0f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:20:05.850152 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1357/2694] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:20:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:20:05.915943 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f0e941-c6f3-4464-8213-e327fb9c906e None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:20:05.916196 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f0e941-c6f3-4464-8213-e327fb9c906e None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:20:05.916472 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f0e941-c6f3-4464-8213-e327fb9c906e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:20:05.939590 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-89f0e941-c6f3-4464-8213-e327fb9c906e None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:20:05.940830 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-89f0e941-c6f3-4464-8213-e327fb9c906e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:20:05.949677 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1338/2695] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:20:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:20:05.953846 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b5cc78-13fc-49c4-8a69-99afa5e0a0f1 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:20:05.963964 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b5cc78-13fc-49c4-8a69-99afa5e0a0f1 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:20:05.970003 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0f9fc2-9cd3-472e-85dd-1ea7fa50ab78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:20:05.970254 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0f9fc2-9cd3-472e-85dd-1ea7fa50ab78 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:20:05.970357 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0f9fc2-9cd3-472e-85dd-1ea7fa50ab78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:20:05.970634 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0f9fc2-9cd3-472e-85dd-1ea7fa50ab78 admin admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:20:05.971340 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0f9fc2-9cd3-472e-85dd-1ea7fa50ab78 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:20:05.971638 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0f9fc2-9cd3-472e-85dd-1ea7fa50ab78 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:20:05.973805 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0f9fc2-9cd3-472e-85dd-1ea7fa50ab78 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:20:05.991709 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f0e941-c6f3-4464-8213-e327fb9c906e None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:20:05.997676 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1358/2696] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 9 12:20:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:20:06.002409 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f0e941-c6f3-4464-8213-e327fb9c906e None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:20:06.009566 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9f033a-6086-4584-ac43-8480d268f486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:20:06.010514 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9f033a-6086-4584-ac43-8480d268f486 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:20:06.011270 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9f033a-6086-4584-ac43-8480d268f486 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:20:06.011921 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9f033a-6086-4584-ac43-8480d268f486 admin admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:20:06.013216 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9f033a-6086-4584-ac43-8480d268f486 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:20:06.016191 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9f033a-6086-4584-ac43-8480d268f486 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:20:06.067211 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1339/2697] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 9 12:20:05 2022] POST /identity/v3/projects => generated 371 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:20:06.068472 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0f9fc2-9cd3-472e-85dd-1ea7fa50ab78 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:20:06.080856 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0f9fc2-9cd3-472e-85dd-1ea7fa50ab78 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:20:06.168719 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafa4b76-de09-4321-a0db-25729b11d48e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:20:06.169462 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafa4b76-de09-4321-a0db-25729b11d48e admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:20:06.169755 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafa4b76-de09-4321-a0db-25729b11d48e admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:20:06.170026 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafa4b76-de09-4321-a0db-25729b11d48e admin admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:20:06.170664 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafa4b76-de09-4321-a0db-25729b11d48e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:20:06.183576 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafa4b76-de09-4321-a0db-25729b11d48e admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:20:06.222254 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1359/2698] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:20:06 2022] POST /identity/v3/users => generated 286 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:20:06.227035 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9f033a-6086-4584-ac43-8480d268f486 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:20:06.241841 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9f033a-6086-4584-ac43-8480d268f486 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:20:06.250730 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a36c7ac-fef9-4bbe-9feb-24c39d74e937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:20:06.251774 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a36c7ac-fef9-4bbe-9feb-24c39d74e937 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:20:06.252750 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a36c7ac-fef9-4bbe-9feb-24c39d74e937 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:20:06.254570 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a36c7ac-fef9-4bbe-9feb-24c39d74e937 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:20:06.256741 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a36c7ac-fef9-4bbe-9feb-24c39d74e937 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:20:06.258570 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a36c7ac-fef9-4bbe-9feb-24c39d74e937 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:20:06.276215 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1340/2699] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:20:06 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:20:06.280061 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafa4b76-de09-4321-a0db-25729b11d48e admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:20:06.322933 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafa4b76-de09-4321-a0db-25729b11d48e admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:20:06.350953 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a3b029-7523-4ddf-8cac-331d760a1028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:20:06.353909 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a3b029-7523-4ddf-8cac-331d760a1028 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:20:06.354190 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a3b029-7523-4ddf-8cac-331d760a1028 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:20:06.354482 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a3b029-7523-4ddf-8cac-331d760a1028 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:20:06.355761 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a3b029-7523-4ddf-8cac-331d760a1028 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:20:06.358140 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a3b029-7523-4ddf-8cac-331d760a1028 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:20:06.377177 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1360/2700] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:20:06 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 12:20:06.379453 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a36c7ac-fef9-4bbe-9feb-24c39d74e937 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:20:06.396155 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a36c7ac-fef9-4bbe-9feb-24c39d74e937 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:20:06.403104 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8ca08a-1d42-4eb6-b9cb-15ff0aef7e53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:20:06.404422 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8ca08a-1d42-4eb6-b9cb-15ff0aef7e53 admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:20:06.404764 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8ca08a-1d42-4eb6-b9cb-15ff0aef7e53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:20:06.405049 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8ca08a-1d42-4eb6-b9cb-15ff0aef7e53 admin admin] PATH_INFO: `/v3/projects/3ccb1d2a36db46e5b16d8ae7f6197c23/users/9e6130026e584b1fa8a3dfac4828452d/roles/eecd2f160dd844f5906f3d345780c501` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:20:06.416196 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:20:06.429397 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8ca08a-1d42-4eb6-b9cb-15ff0aef7e53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ccb1d2a36db46e5b16d8ae7f6197c23, user_id=9e6130026e584b1fa8a3dfac4828452d, role_id=eecd2f160dd844f5906f3d345780c501)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:20:06.437370 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8ca08a-1d42-4eb6-b9cb-15ff0aef7e53 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:20:06.455096 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1341/2701] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:20:06 2022] PUT /identity/v3/projects/3ccb1d2a36db46e5b16d8ae7f6197c23/users/9e6130026e584b1fa8a3dfac4828452d/roles/eecd2f160dd844f5906f3d345780c501 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:20:06.468296 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbc15bd-8316-4a53-850f-2d8fe20bf451 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:20:06.469235 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbc15bd-8316-4a53-850f-2d8fe20bf451 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:20:06.469853 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbc15bd-8316-4a53-850f-2d8fe20bf451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:20:06.507507 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-cbbc15bd-8316-4a53-850f-2d8fe20bf451 None None] MFA Rules not processed for user `9e6130026e584b1fa8a3dfac4828452d`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:20:06.508913 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-cbbc15bd-8316-4a53-850f-2d8fe20bf451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:20:06.542843 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:20:06.621931 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1361/2702] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:20:06 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:20:06.636607 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8ca08a-1d42-4eb6-b9cb-15ff0aef7e53 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:20:06.638663 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-ab8ca08a-1d42-4eb6-b9cb-15ff0aef7e53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:20:06.757205 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8ca08a-1d42-4eb6-b9cb-15ff0aef7e53 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:20:06.764464 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ed5ff0-57c9-4e14-9a6a-d4a9543a2292 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37c4e14aa8e04d3e85a406b9363bca2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:20:06.765401 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ed5ff0-57c9-4e14-9a6a-d4a9543a2292 service glance] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:20:06.765913 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ed5ff0-57c9-4e14-9a6a-d4a9543a2292 service glance] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:20:06.766506 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ed5ff0-57c9-4e14-9a6a-d4a9543a2292 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:20:06.781762 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ed5ff0-57c9-4e14-9a6a-d4a9543a2292 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:20:06.784581 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ed5ff0-57c9-4e14-9a6a-d4a9543a2292 service glance] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:20:06.797748 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1342/2703] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:20:06 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:20:06.912706 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbc15bd-8316-4a53-850f-2d8fe20bf451 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:20:06.914647 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-cbbc15bd-8316-4a53-850f-2d8fe20bf451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:20:06.976615 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ed5ff0-57c9-4e14-9a6a-d4a9543a2292 service glance] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:20:06.978440 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-f7ed5ff0-57c9-4e14-9a6a-d4a9543a2292 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:20:07.026267 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbc15bd-8316-4a53-850f-2d8fe20bf451 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:20:07.033040 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0f902f-52da-406c-8c29-cfd8b8a0cab6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3adb2f0583c4ff782049dacdb3ee556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:20:07.033896 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0f902f-52da-406c-8c29-cfd8b8a0cab6 service placement] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:20:07.034178 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0f902f-52da-406c-8c29-cfd8b8a0cab6 service placement] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:20:07.034447 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0f902f-52da-406c-8c29-cfd8b8a0cab6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:20:07.036878 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-2d0f902f-52da-406c-8c29-cfd8b8a0cab6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:20:07.050051 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0f902f-52da-406c-8c29-cfd8b8a0cab6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:20:07.053377 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0f902f-52da-406c-8c29-cfd8b8a0cab6 service placement] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:20:07.056771 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ed5ff0-57c9-4e14-9a6a-d4a9543a2292 service glance] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:20:07.061925 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd739f8-3b49-4f9b-9bd3-5525e8cf275a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:20:07.062558 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd739f8-3b49-4f9b-9bd3-5525e8cf275a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:20:07.063161 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd739f8-3b49-4f9b-9bd3-5525e8cf275a admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:20:07.063440 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd739f8-3b49-4f9b-9bd3-5525e8cf275a admin admin] PATH_INFO: `/v3/users/b9a5c9fb3f664ae097f0c3ce6f6c0764` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:20:07.064918 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd739f8-3b49-4f9b-9bd3-5525e8cf275a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9a5c9fb3f664ae097f0c3ce6f6c0764)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:20:07.065218 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1362/2704] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:20:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:20:07.067472 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd739f8-3b49-4f9b-9bd3-5525e8cf275a admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:20:07.122116 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-6dd739f8-3b49-4f9b-9bd3-5525e8cf275a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9a5c9fb3f664ae097f0c3ce6f6c0764'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:20:07.136905 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-6dd739f8-3b49-4f9b-9bd3-5525e8cf275a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9a5c9fb3f664ae097f0c3ce6f6c0764'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:20:07.144980 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-6dd739f8-3b49-4f9b-9bd3-5525e8cf275a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9a5c9fb3f664ae097f0c3ce6f6c0764'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:20:07.153996 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-6dd739f8-3b49-4f9b-9bd3-5525e8cf275a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9a5c9fb3f664ae097f0c3ce6f6c0764'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:20:07.154734 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-6dd739f8-3b49-4f9b-9bd3-5525e8cf275a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9a5c9fb3f664ae097f0c3ce6f6c0764'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:20:07.162489 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1343/2705] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 9 12:20:06 2022] DELETE /identity/v3/users/b9a5c9fb3f664ae097f0c3ce6f6c0764 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:20:07.360578 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0f902f-52da-406c-8c29-cfd8b8a0cab6 service placement] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:20:07.375241 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0f902f-52da-406c-8c29-cfd8b8a0cab6 service placement] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:20:07.438834 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9037c240-3406-43dd-84fe-0111f3db209f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:20:07.439483 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9037c240-3406-43dd-84fe-0111f3db209f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:20:07.439763 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9037c240-3406-43dd-84fe-0111f3db209f admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:20:07.440068 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9037c240-3406-43dd-84fe-0111f3db209f admin admin] PATH_INFO: `/v3/projects/f2f72b6ae8d3482e935d3375bc999732` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:20:07.441746 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9037c240-3406-43dd-84fe-0111f3db209f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2f72b6ae8d3482e935d3375bc999732)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:20:07.443961 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9037c240-3406-43dd-84fe-0111f3db209f admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:20:07.486188 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:20:07.490192 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-9037c240-3406-43dd-84fe-0111f3db209f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2f72b6ae8d3482e935d3375bc999732', 'request_id': 'req-9037c240-3406-43dd-84fe-0111f3db209f'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:20:07.496042 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-9037c240-3406-43dd-84fe-0111f3db209f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2f72b6ae8d3482e935d3375bc999732', 'request_id': 'req-9037c240-3406-43dd-84fe-0111f3db209f'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:20:07.502720 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-9037c240-3406-43dd-84fe-0111f3db209f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2f72b6ae8d3482e935d3375bc999732', 'request_id': 'req-9037c240-3406-43dd-84fe-0111f3db209f'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:20:07.516119 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-9037c240-3406-43dd-84fe-0111f3db209f admin admin] The token cache is being invalidate because project f2f72b6ae8d3482e935d3375bc999732 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81591) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:20:07.516405 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-9037c240-3406-43dd-84fe-0111f3db209f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:20:07.517149 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-9037c240-3406-43dd-84fe-0111f3db209f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:20:07.518704 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1363/2706] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 9 12:20:07 2022] DELETE /identity/v3/projects/f2f72b6ae8d3482e935d3375bc999732 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:20:07.524619 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d6f6e9-99be-4126-bfb1-8f7cb7e2add2 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:20:07.524900 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d6f6e9-99be-4126-bfb1-8f7cb7e2add2 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:20:07.525178 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d6f6e9-99be-4126-bfb1-8f7cb7e2add2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:20:07.557913 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-e6d6f6e9-99be-4126-bfb1-8f7cb7e2add2 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:20:07.559235 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-e6d6f6e9-99be-4126-bfb1-8f7cb7e2add2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:20:07.564071 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:20:07.631853 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1344/2707] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:20:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:20:07.640018 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37826b51-8bee-4193-b4f9-3c9f48291e82 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:20:07.640333 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37826b51-8bee-4193-b4f9-3c9f48291e82 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:20:07.640608 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37826b51-8bee-4193-b4f9-3c9f48291e82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:20:07.665452 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-37826b51-8bee-4193-b4f9-3c9f48291e82 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:20:07.666731 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-37826b51-8bee-4193-b4f9-3c9f48291e82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:20:07.676879 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1364/2708] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:20:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:20:07.681416 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d6f6e9-99be-4126-bfb1-8f7cb7e2add2 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:20:07.693822 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d6f6e9-99be-4126-bfb1-8f7cb7e2add2 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:20:07.698839 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb4dddf-8664-4a22-8229-4e5679efbf0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:20:07.699722 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb4dddf-8664-4a22-8229-4e5679efbf0e admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:20:07.700061 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb4dddf-8664-4a22-8229-4e5679efbf0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:20:07.700537 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb4dddf-8664-4a22-8229-4e5679efbf0e admin admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:20:07.701262 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb4dddf-8664-4a22-8229-4e5679efbf0e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:20:07.701753 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb4dddf-8664-4a22-8229-4e5679efbf0e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:20:07.703733 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb4dddf-8664-4a22-8229-4e5679efbf0e admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:20:07.716245 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1345/2709] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 9 12:20:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:20:07.720656 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37826b51-8bee-4193-b4f9-3c9f48291e82 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:20:07.732187 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37826b51-8bee-4193-b4f9-3c9f48291e82 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:20:07.737014 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3822ce79-eac7-480c-befd-f945d87c0c3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:20:07.738407 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3822ce79-eac7-480c-befd-f945d87c0c3d admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:20:07.738407 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3822ce79-eac7-480c-befd-f945d87c0c3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:20:07.738777 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3822ce79-eac7-480c-befd-f945d87c0c3d admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:20:07.739960 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3822ce79-eac7-480c-befd-f945d87c0c3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:20:07.741847 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3822ce79-eac7-480c-befd-f945d87c0c3d admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:20:07.775728 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1365/2710] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 9 12:20:07 2022] POST /identity/v3/projects => generated 377 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:20:07.779665 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb4dddf-8664-4a22-8229-4e5679efbf0e admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:20:07.790836 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb4dddf-8664-4a22-8229-4e5679efbf0e admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:20:07.850693 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3822ce79-eac7-480c-befd-f945d87c0c3d admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:20:07.853181 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-3822ce79-eac7-480c-befd-f945d87c0c3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:20:07.937978 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3822ce79-eac7-480c-befd-f945d87c0c3d admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:20:07.942355 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e362c602-f756-4107-9e26-42c3811ae007 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:20:07.943109 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e362c602-f756-4107-9e26-42c3811ae007 service neutron] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:20:07.943437 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e362c602-f756-4107-9e26-42c3811ae007 service neutron] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:20:07.943782 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e362c602-f756-4107-9e26-42c3811ae007 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:20:07.945633 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-e362c602-f756-4107-9e26-42c3811ae007 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:20:07.992319 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7b13b5-d07f-4460-9ad5-b67734fd49f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:20:07.992941 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7b13b5-d07f-4460-9ad5-b67734fd49f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:20:07.993219 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7b13b5-d07f-4460-9ad5-b67734fd49f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:20:07.993490 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7b13b5-d07f-4460-9ad5-b67734fd49f1 admin admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:20:07.994208 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7b13b5-d07f-4460-9ad5-b67734fd49f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:20:07.996220 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7b13b5-d07f-4460-9ad5-b67734fd49f1 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:20:08.014150 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e362c602-f756-4107-9e26-42c3811ae007 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:20:08.015632 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e362c602-f756-4107-9e26-42c3811ae007 service neutron] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:20:08.021636 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1346/2711] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:20:07 2022] POST /identity/v3/users => generated 289 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:20:08.022686 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7b13b5-d07f-4460-9ad5-b67734fd49f1 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:20:08.026529 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1366/2712] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:20:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:20:08.033724 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7b13b5-d07f-4460-9ad5-b67734fd49f1 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:20:08.040677 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2a77b2-a8e2-44c2-b134-c1d9e9ff7cd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:20:08.041442 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2a77b2-a8e2-44c2-b134-c1d9e9ff7cd2 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:20:08.041743 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2a77b2-a8e2-44c2-b134-c1d9e9ff7cd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:20:08.042008 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2a77b2-a8e2-44c2-b134-c1d9e9ff7cd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:20:08.042645 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2a77b2-a8e2-44c2-b134-c1d9e9ff7cd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:20:08.044276 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2a77b2-a8e2-44c2-b134-c1d9e9ff7cd2 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:20:08.056511 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1347/2713] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:20:08 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 12:20:08.060471 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e362c602-f756-4107-9e26-42c3811ae007 service neutron] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:20:08.070655 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e362c602-f756-4107-9e26-42c3811ae007 service neutron] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:20:08.075441 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e5e87e-ed82-4958-888a-055ac39c3fdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:20:08.076054 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e5e87e-ed82-4958-888a-055ac39c3fdd admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:20:08.076335 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e5e87e-ed82-4958-888a-055ac39c3fdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:20:08.076657 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e5e87e-ed82-4958-888a-055ac39c3fdd admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:20:08.077272 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e5e87e-ed82-4958-888a-055ac39c3fdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:20:08.078856 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e5e87e-ed82-4958-888a-055ac39c3fdd admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:20:08.089547 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1367/2714] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:20:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:20:08.093837 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2a77b2-a8e2-44c2-b134-c1d9e9ff7cd2 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:20:08.104599 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2a77b2-a8e2-44c2-b134-c1d9e9ff7cd2 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:20:08.109220 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d43fe2-7209-405a-9487-91b321d00ee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:20:08.109877 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d43fe2-7209-405a-9487-91b321d00ee1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:20:08.110155 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d43fe2-7209-405a-9487-91b321d00ee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:20:08.110426 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d43fe2-7209-405a-9487-91b321d00ee1 admin admin] PATH_INFO: `/v3/projects/69e19a94fd744d9e94d9f5ff5b089f15/users/870f5366f958496d93c51e4b573e2487/roles/eecd2f160dd844f5906f3d345780c501` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:20:08.128985 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d43fe2-7209-405a-9487-91b321d00ee1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69e19a94fd744d9e94d9f5ff5b089f15, user_id=870f5366f958496d93c51e4b573e2487, role_id=eecd2f160dd844f5906f3d345780c501)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:20:08.132072 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d43fe2-7209-405a-9487-91b321d00ee1 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:20:08.150426 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1348/2715] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:20:08 2022] PUT /identity/v3/projects/69e19a94fd744d9e94d9f5ff5b089f15/users/870f5366f958496d93c51e4b573e2487/roles/eecd2f160dd844f5906f3d345780c501 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:20:08.155679 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a899ad-0744-47f4-9afb-c9d0375d62f6 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:20:08.155961 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a899ad-0744-47f4-9afb-c9d0375d62f6 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:20:08.156226 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a899ad-0744-47f4-9afb-c9d0375d62f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:20:08.180228 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-87a899ad-0744-47f4-9afb-c9d0375d62f6 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:20:08.181437 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-87a899ad-0744-47f4-9afb-c9d0375d62f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:20:08.225132 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:DELETE=4,keystone:SELECT=94,keystone:INSERT=2 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:20:08.246473 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1368/2716] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:20:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:20:08.259195 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d43fe2-7209-405a-9487-91b321d00ee1 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:20:08.260938 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-77d43fe2-7209-405a-9487-91b321d00ee1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:20:08.323532 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d43fe2-7209-405a-9487-91b321d00ee1 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:20:08.327704 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a929fa-f28d-4730-9f8a-ef00f1c1ceed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:20:08.328506 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a929fa-f28d-4730-9f8a-ef00f1c1ceed service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:20:08.328981 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a929fa-f28d-4730-9f8a-ef00f1c1ceed service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:20:08.329310 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a929fa-f28d-4730-9f8a-ef00f1c1ceed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:20:08.336989 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:20:08.341074 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a929fa-f28d-4730-9f8a-ef00f1c1ceed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:20:08.343069 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a929fa-f28d-4730-9f8a-ef00f1c1ceed service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:20:08.354164 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1349/2717] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:20:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:20:15.699899 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910bbf4b-0391-468d-80fd-eace2195b876 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:20:15.700881 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910bbf4b-0391-468d-80fd-eace2195b876 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:20:15.702056 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910bbf4b-0391-468d-80fd-eace2195b876 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:20:15.739126 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-910bbf4b-0391-468d-80fd-eace2195b876 None None] MFA Rules not processed for user `870f5366f958496d93c51e4b573e2487`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:20:15.741313 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-910bbf4b-0391-468d-80fd-eace2195b876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:20:15.871890 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1369/2718] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:20:15 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:20:15.879521 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a929fa-f28d-4730-9f8a-ef00f1c1ceed service neutron] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:20:15.881006 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-36a929fa-f28d-4730-9f8a-ef00f1c1ceed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:20:15.989832 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a929fa-f28d-4730-9f8a-ef00f1c1ceed service neutron] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:20:16.008221 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f661bb5-d678-4e0c-b432-04ca389b9f85 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70bac169d5cd43e1926f310344ac11fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:20:16.009050 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f661bb5-d678-4e0c-b432-04ca389b9f85 service nova] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:20:16.009224 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f661bb5-d678-4e0c-b432-04ca389b9f85 service nova] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:20:16.009494 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f661bb5-d678-4e0c-b432-04ca389b9f85 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:20:16.020893 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f661bb5-d678-4e0c-b432-04ca389b9f85 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:20:16.022845 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f661bb5-d678-4e0c-b432-04ca389b9f85 service nova] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:20:16.035229 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1350/2719] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:20:15 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:20:17.866367 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910bbf4b-0391-468d-80fd-eace2195b876 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:20:17.868706 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-910bbf4b-0391-468d-80fd-eace2195b876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:20:17.889745 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910bbf4b-0391-468d-80fd-eace2195b876 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:20:17.895458 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5d06a3-ae9d-4b1a-a236-ad22c2c9448f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:20:17.896634 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5d06a3-ae9d-4b1a-a236-ad22c2c9448f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:20:17.897227 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5d06a3-ae9d-4b1a-a236-ad22c2c9448f admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:20:17.897501 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5d06a3-ae9d-4b1a-a236-ad22c2c9448f admin admin] PATH_INFO: `/v3/users/9750f671d674447f9fbf36d382573c37` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:20:17.898699 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5d06a3-ae9d-4b1a-a236-ad22c2c9448f admin admin] RBAC: Authorizing `identity:delete_user(user_id=9750f671d674447f9fbf36d382573c37)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:20:17.900704 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5d06a3-ae9d-4b1a-a236-ad22c2c9448f admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:20:17.963773 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-ca5d06a3-ae9d-4b1a-a236-ad22c2c9448f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9750f671d674447f9fbf36d382573c37'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:20:17.972678 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-ca5d06a3-ae9d-4b1a-a236-ad22c2c9448f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9750f671d674447f9fbf36d382573c37'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:20:17.986631 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-ca5d06a3-ae9d-4b1a-a236-ad22c2c9448f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9750f671d674447f9fbf36d382573c37'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:20:17.996308 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-ca5d06a3-ae9d-4b1a-a236-ad22c2c9448f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9750f671d674447f9fbf36d382573c37'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:20:17.997188 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-ca5d06a3-ae9d-4b1a-a236-ad22c2c9448f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9750f671d674447f9fbf36d382573c37'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:20:18.006197 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1370/2720] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 9 12:20:17 2022] DELETE /identity/v3/users/9750f671d674447f9fbf36d382573c37 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:20:18.243570 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f661bb5-d678-4e0c-b432-04ca389b9f85 service nova] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:20:18.244821 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dde2e1-ec9c-4fc7-86e8-d32f818b7ca1 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:20:18.245354 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dde2e1-ec9c-4fc7-86e8-d32f818b7ca1 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:20:18.245804 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dde2e1-ec9c-4fc7-86e8-d32f818b7ca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:20:18.266551 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f661bb5-d678-4e0c-b432-04ca389b9f85 service nova] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:20:18.277191 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-16dde2e1-ec9c-4fc7-86e8-d32f818b7ca1 None None] MFA Rules not processed for user `870f5366f958496d93c51e4b573e2487`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:20:18.278528 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-16dde2e1-ec9c-4fc7-86e8-d32f818b7ca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:20:18.331763 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:20:18.350524 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1371/2721] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:20:18 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:20:18.354832 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec58841-096e-44aa-abf6-1be53574eb37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:20:18.355561 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec58841-096e-44aa-abf6-1be53574eb37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:20:18.356176 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec58841-096e-44aa-abf6-1be53574eb37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:20:18.356610 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec58841-096e-44aa-abf6-1be53574eb37 admin admin] PATH_INFO: `/v3/projects/6b0570ae08fc40ccb3dd942591098d34` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:20:18.358047 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec58841-096e-44aa-abf6-1be53574eb37 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b0570ae08fc40ccb3dd942591098d34)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:20:18.361650 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec58841-096e-44aa-abf6-1be53574eb37 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:20:18.415540 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-7ec58841-096e-44aa-abf6-1be53574eb37 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b0570ae08fc40ccb3dd942591098d34', 'request_id': 'req-7ec58841-096e-44aa-abf6-1be53574eb37'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:20:18.421927 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-7ec58841-096e-44aa-abf6-1be53574eb37 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b0570ae08fc40ccb3dd942591098d34', 'request_id': 'req-7ec58841-096e-44aa-abf6-1be53574eb37'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:20:18.431695 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-7ec58841-096e-44aa-abf6-1be53574eb37 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b0570ae08fc40ccb3dd942591098d34', 'request_id': 'req-7ec58841-096e-44aa-abf6-1be53574eb37'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:20:18.444826 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-7ec58841-096e-44aa-abf6-1be53574eb37 admin admin] The token cache is being invalidate because project 6b0570ae08fc40ccb3dd942591098d34 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:20:18.445119 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-7ec58841-096e-44aa-abf6-1be53574eb37 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:20:18.445787 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-7ec58841-096e-44aa-abf6-1be53574eb37 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:20:18.447999 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1351/2722] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 9 12:20:18 2022] DELETE /identity/v3/projects/6b0570ae08fc40ccb3dd942591098d34 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:20:18.452067 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dde2e1-ec9c-4fc7-86e8-d32f818b7ca1 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:20:18.453472 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-16dde2e1-ec9c-4fc7-86e8-d32f818b7ca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:20:18.468401 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dde2e1-ec9c-4fc7-86e8-d32f818b7ca1 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:20:18.538197 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab92ca4-868f-4acd-aa7b-b075e79deee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:20:18.538845 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab92ca4-868f-4acd-aa7b-b075e79deee9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:20:18.539159 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab92ca4-868f-4acd-aa7b-b075e79deee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:20:18.539429 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab92ca4-868f-4acd-aa7b-b075e79deee9 admin admin] PATH_INFO: `/v3/users/9039b37485b44800b2aff38093793ccb` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:20:18.540598 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab92ca4-868f-4acd-aa7b-b075e79deee9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9039b37485b44800b2aff38093793ccb)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:20:18.542552 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab92ca4-868f-4acd-aa7b-b075e79deee9 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:20:18.602615 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-2ab92ca4-868f-4acd-aa7b-b075e79deee9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9039b37485b44800b2aff38093793ccb'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:20:18.610674 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-2ab92ca4-868f-4acd-aa7b-b075e79deee9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9039b37485b44800b2aff38093793ccb'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:20:18.632245 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-2ab92ca4-868f-4acd-aa7b-b075e79deee9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9039b37485b44800b2aff38093793ccb'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:20:18.649262 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-2ab92ca4-868f-4acd-aa7b-b075e79deee9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9039b37485b44800b2aff38093793ccb'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:20:18.650397 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-2ab92ca4-868f-4acd-aa7b-b075e79deee9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9039b37485b44800b2aff38093793ccb'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:20:18.664064 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1372/2723] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 9 12:20:18 2022] DELETE /identity/v3/users/9039b37485b44800b2aff38093793ccb => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:20:18.859570 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec58841-096e-44aa-abf6-1be53574eb37 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:20:18.861240 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-7ec58841-096e-44aa-abf6-1be53574eb37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:20:18.883894 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec58841-096e-44aa-abf6-1be53574eb37 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:20:18.888665 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:20:18.964363 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb24cc6-cfb1-4460-928d-3a4a3d16111c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:20:18.965035 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb24cc6-cfb1-4460-928d-3a4a3d16111c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:20:18.965310 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb24cc6-cfb1-4460-928d-3a4a3d16111c admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:20:18.965565 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb24cc6-cfb1-4460-928d-3a4a3d16111c admin admin] PATH_INFO: `/v3/projects/2111959fe7e94015bd8f0383ab5c6f46` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:20:18.968429 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb24cc6-cfb1-4460-928d-3a4a3d16111c admin admin] RBAC: Authorizing `identity:delete_project(project_id=2111959fe7e94015bd8f0383ab5c6f46)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:20:18.970936 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb24cc6-cfb1-4460-928d-3a4a3d16111c admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:20:19.042324 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-6bb24cc6-cfb1-4460-928d-3a4a3d16111c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2111959fe7e94015bd8f0383ab5c6f46', 'request_id': 'req-6bb24cc6-cfb1-4460-928d-3a4a3d16111c'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:20:19.049361 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-6bb24cc6-cfb1-4460-928d-3a4a3d16111c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2111959fe7e94015bd8f0383ab5c6f46', 'request_id': 'req-6bb24cc6-cfb1-4460-928d-3a4a3d16111c'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:20:19.059582 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-6bb24cc6-cfb1-4460-928d-3a4a3d16111c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2111959fe7e94015bd8f0383ab5c6f46', 'request_id': 'req-6bb24cc6-cfb1-4460-928d-3a4a3d16111c'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:20:19.080210 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-6bb24cc6-cfb1-4460-928d-3a4a3d16111c admin admin] The token cache is being invalidate because project 2111959fe7e94015bd8f0383ab5c6f46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:20:19.080339 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-6bb24cc6-cfb1-4460-928d-3a4a3d16111c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:20:19.081130 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-6bb24cc6-cfb1-4460-928d-3a4a3d16111c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:20:19.082759 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1352/2724] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 9 12:20:18 2022] DELETE /identity/v3/projects/2111959fe7e94015bd8f0383ab5c6f46 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:20:28.656247 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=57,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:20:29.077213 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=47,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:20:30.917794 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab92ca4-868f-4acd-aa7b-b075e79deee9 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:20:30.919385 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-2ab92ca4-868f-4acd-aa7b-b075e79deee9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:20:30.933244 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab92ca4-868f-4acd-aa7b-b075e79deee9 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:20:30.990027 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a287a6a6-753f-4ccf-90fd-fe024c6ac48b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:20:30.990650 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a287a6a6-753f-4ccf-90fd-fe024c6ac48b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:20:30.990951 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a287a6a6-753f-4ccf-90fd-fe024c6ac48b admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:20:30.991240 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a287a6a6-753f-4ccf-90fd-fe024c6ac48b admin admin] PATH_INFO: `/v3/users/9e6130026e584b1fa8a3dfac4828452d` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:20:30.992358 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a287a6a6-753f-4ccf-90fd-fe024c6ac48b admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e6130026e584b1fa8a3dfac4828452d)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:20:30.994255 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a287a6a6-753f-4ccf-90fd-fe024c6ac48b admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:20:31.039369 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-a287a6a6-753f-4ccf-90fd-fe024c6ac48b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e6130026e584b1fa8a3dfac4828452d'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:20:31.046094 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-a287a6a6-753f-4ccf-90fd-fe024c6ac48b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e6130026e584b1fa8a3dfac4828452d'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:20:31.085024 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-a287a6a6-753f-4ccf-90fd-fe024c6ac48b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e6130026e584b1fa8a3dfac4828452d'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:20:31.093320 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-a287a6a6-753f-4ccf-90fd-fe024c6ac48b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e6130026e584b1fa8a3dfac4828452d'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:20:31.093940 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-a287a6a6-753f-4ccf-90fd-fe024c6ac48b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e6130026e584b1fa8a3dfac4828452d'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:20:31.100436 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1373/2725] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 9 12:20:30 2022] DELETE /identity/v3/users/9e6130026e584b1fa8a3dfac4828452d => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:20:31.105063 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfb9b1e-07f2-45f1-9598-078e181e6fa2 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:20:31.105540 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfb9b1e-07f2-45f1-9598-078e181e6fa2 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:20:31.105641 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfb9b1e-07f2-45f1-9598-078e181e6fa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:20:31.123659 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-fbfb9b1e-07f2-45f1-9598-078e181e6fa2 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:20:31.124838 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-fbfb9b1e-07f2-45f1-9598-078e181e6fa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:20:31.182427 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1353/2726] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:20:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:20:31.192406 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a287a6a6-753f-4ccf-90fd-fe024c6ac48b admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:20:31.193816 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-a287a6a6-753f-4ccf-90fd-fe024c6ac48b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:20:31.250852 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a287a6a6-753f-4ccf-90fd-fe024c6ac48b admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:20:31.255248 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7ae17f-b75e-4b76-b80b-b24fe3b99233 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:20:31.255843 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7ae17f-b75e-4b76-b80b-b24fe3b99233 service neutron] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:20:31.256118 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7ae17f-b75e-4b76-b80b-b24fe3b99233 service neutron] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:20:31.256386 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7ae17f-b75e-4b76-b80b-b24fe3b99233 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:20:31.265142 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7ae17f-b75e-4b76-b80b-b24fe3b99233 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:20:31.266550 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7ae17f-b75e-4b76-b80b-b24fe3b99233 service neutron] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:20:31.276607 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1374/2727] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:20:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:20:31.544269 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfb9b1e-07f2-45f1-9598-078e181e6fa2 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:20:31.545748 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-fbfb9b1e-07f2-45f1-9598-078e181e6fa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:20:31.558085 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfb9b1e-07f2-45f1-9598-078e181e6fa2 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:20:31.562721 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eca7c2b-299d-4b39-8407-2b45598c8411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:20:31.563414 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eca7c2b-299d-4b39-8407-2b45598c8411 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:20:31.563695 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eca7c2b-299d-4b39-8407-2b45598c8411 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:20:31.563965 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eca7c2b-299d-4b39-8407-2b45598c8411 admin admin] PATH_INFO: `/v3/projects/3ccb1d2a36db46e5b16d8ae7f6197c23` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:20:31.565619 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eca7c2b-299d-4b39-8407-2b45598c8411 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ccb1d2a36db46e5b16d8ae7f6197c23)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:20:31.567701 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eca7c2b-299d-4b39-8407-2b45598c8411 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:20:31.616311 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-3eca7c2b-299d-4b39-8407-2b45598c8411 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ccb1d2a36db46e5b16d8ae7f6197c23', 'request_id': 'req-3eca7c2b-299d-4b39-8407-2b45598c8411'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:20:31.621865 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-3eca7c2b-299d-4b39-8407-2b45598c8411 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ccb1d2a36db46e5b16d8ae7f6197c23', 'request_id': 'req-3eca7c2b-299d-4b39-8407-2b45598c8411'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:20:31.630062 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-3eca7c2b-299d-4b39-8407-2b45598c8411 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ccb1d2a36db46e5b16d8ae7f6197c23', 'request_id': 'req-3eca7c2b-299d-4b39-8407-2b45598c8411'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:20:31.644256 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-3eca7c2b-299d-4b39-8407-2b45598c8411 admin admin] The token cache is being invalidate because project 3ccb1d2a36db46e5b16d8ae7f6197c23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:20:31.644256 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-3eca7c2b-299d-4b39-8407-2b45598c8411 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:20:31.644256 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-3eca7c2b-299d-4b39-8407-2b45598c8411 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:20:31.644256 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1354/2728] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 9 12:20:31 2022] DELETE /identity/v3/projects/3ccb1d2a36db46e5b16d8ae7f6197c23 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:20:31.649783 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c5aaf7-30eb-45ad-a8fd-09889ffb820a None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:20:31.650068 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c5aaf7-30eb-45ad-a8fd-09889ffb820a None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:20:31.650339 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c5aaf7-30eb-45ad-a8fd-09889ffb820a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:20:31.669342 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-43c5aaf7-30eb-45ad-a8fd-09889ffb820a None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:20:31.670540 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-43c5aaf7-30eb-45ad-a8fd-09889ffb820a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:20:31.688662 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:20:31.731495 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1375/2729] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:20:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:20:31.740072 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3f5872-bf70-47c3-a63b-f1983b9d1b57 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:20:31.740382 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3f5872-bf70-47c3-a63b-f1983b9d1b57 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:20:31.740659 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3f5872-bf70-47c3-a63b-f1983b9d1b57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:20:31.760677 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-1b3f5872-bf70-47c3-a63b-f1983b9d1b57 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:20:31.761873 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-1b3f5872-bf70-47c3-a63b-f1983b9d1b57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:20:31.769821 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1355/2730] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:20:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:20:31.773982 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c5aaf7-30eb-45ad-a8fd-09889ffb820a None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:20:31.783410 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c5aaf7-30eb-45ad-a8fd-09889ffb820a None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:20:31.788015 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7dfe1a-56ff-495e-8a62-436ab1914ffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:20:31.788605 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7dfe1a-56ff-495e-8a62-436ab1914ffa admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:20:31.788879 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7dfe1a-56ff-495e-8a62-436ab1914ffa admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:20:31.789143 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7dfe1a-56ff-495e-8a62-436ab1914ffa admin admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:20:31.789776 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7dfe1a-56ff-495e-8a62-436ab1914ffa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:20:31.790075 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7dfe1a-56ff-495e-8a62-436ab1914ffa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:20:31.791800 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7dfe1a-56ff-495e-8a62-436ab1914ffa admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:20:31.804512 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1376/2731] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 9 12:20:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:20:31.809057 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3f5872-bf70-47c3-a63b-f1983b9d1b57 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:20:31.819161 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3f5872-bf70-47c3-a63b-f1983b9d1b57 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:20:31.823878 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb1519e-d00d-4d0f-a48a-1dcbb749275a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:20:31.824474 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb1519e-d00d-4d0f-a48a-1dcbb749275a admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:20:31.824752 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb1519e-d00d-4d0f-a48a-1dcbb749275a admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:20:31.825024 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb1519e-d00d-4d0f-a48a-1dcbb749275a admin admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:20:31.825705 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb1519e-d00d-4d0f-a48a-1dcbb749275a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:20:31.827646 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb1519e-d00d-4d0f-a48a-1dcbb749275a admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:20:31.854312 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1356/2732] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 9 12:20:31 2022] POST /identity/v3/projects => generated 375 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:20:31.858733 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7dfe1a-56ff-495e-8a62-436ab1914ffa admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:20:31.869251 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7dfe1a-56ff-495e-8a62-436ab1914ffa admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:20:31.925038 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c82353a-bb48-4075-9b31-df18b1d8ea66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:20:31.925928 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c82353a-bb48-4075-9b31-df18b1d8ea66 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:20:31.926155 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c82353a-bb48-4075-9b31-df18b1d8ea66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:20:31.926423 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c82353a-bb48-4075-9b31-df18b1d8ea66 admin admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:20:31.927079 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c82353a-bb48-4075-9b31-df18b1d8ea66 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:20:31.928981 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c82353a-bb48-4075-9b31-df18b1d8ea66 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:20:31.950375 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1377/2733] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:20:31 2022] POST /identity/v3/users => generated 288 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:20:31.954582 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb1519e-d00d-4d0f-a48a-1dcbb749275a admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:20:31.964934 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb1519e-d00d-4d0f-a48a-1dcbb749275a admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:20:31.969502 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88216bf-b833-47f0-b759-e65b9afa1827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:20:31.970095 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88216bf-b833-47f0-b759-e65b9afa1827 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:20:31.970370 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88216bf-b833-47f0-b759-e65b9afa1827 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:20:31.970653 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88216bf-b833-47f0-b759-e65b9afa1827 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:20:31.971659 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88216bf-b833-47f0-b759-e65b9afa1827 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:20:31.973251 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88216bf-b833-47f0-b759-e65b9afa1827 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:20:31.983292 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1357/2734] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:20:31 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:20:31.987908 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c82353a-bb48-4075-9b31-df18b1d8ea66 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:20:31.997734 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c82353a-bb48-4075-9b31-df18b1d8ea66 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:20:32.003024 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab6f7ae-a0c1-4bcf-9a22-a2d190d88036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:20:32.003529 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab6f7ae-a0c1-4bcf-9a22-a2d190d88036 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:20:32.003808 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab6f7ae-a0c1-4bcf-9a22-a2d190d88036 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:20:32.004076 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab6f7ae-a0c1-4bcf-9a22-a2d190d88036 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:20:32.005116 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab6f7ae-a0c1-4bcf-9a22-a2d190d88036 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:20:32.006698 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab6f7ae-a0c1-4bcf-9a22-a2d190d88036 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:20:32.017145 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1378/2735] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:20:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:20:32.021156 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88216bf-b833-47f0-b759-e65b9afa1827 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:20:32.031043 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88216bf-b833-47f0-b759-e65b9afa1827 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:20:32.036276 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8ade81-e1f8-4d60-9232-2e1322b007c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:20:32.036913 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8ade81-e1f8-4d60-9232-2e1322b007c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:20:32.037190 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8ade81-e1f8-4d60-9232-2e1322b007c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:20:32.037462 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8ade81-e1f8-4d60-9232-2e1322b007c1 admin admin] PATH_INFO: `/v3/projects/192b3c1dcb7a4d9c952420130039ea6c/users/c0781112ec0c440da34254add94a593a/roles/eecd2f160dd844f5906f3d345780c501` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:20:32.052494 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8ade81-e1f8-4d60-9232-2e1322b007c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=192b3c1dcb7a4d9c952420130039ea6c, user_id=c0781112ec0c440da34254add94a593a, role_id=eecd2f160dd844f5906f3d345780c501)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:20:32.055177 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8ade81-e1f8-4d60-9232-2e1322b007c1 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:20:32.067624 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1358/2736] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:20:32 2022] PUT /identity/v3/projects/192b3c1dcb7a4d9c952420130039ea6c/users/c0781112ec0c440da34254add94a593a/roles/eecd2f160dd844f5906f3d345780c501 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:20:32.079227 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd08ced-014b-49bc-94f7-f42875efcff8 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:20:32.079592 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd08ced-014b-49bc-94f7-f42875efcff8 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:20:32.079865 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd08ced-014b-49bc-94f7-f42875efcff8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:20:32.107979 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-fcd08ced-014b-49bc-94f7-f42875efcff8 None None] MFA Rules not processed for user `c0781112ec0c440da34254add94a593a`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:20:32.109235 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-fcd08ced-014b-49bc-94f7-f42875efcff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:20:32.171342 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1379/2737] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:20:32 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:20:32.183105 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8ade81-e1f8-4d60-9232-2e1322b007c1 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:20:32.184807 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-6c8ade81-e1f8-4d60-9232-2e1322b007c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:20:32.206517 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:20:32.244586 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8ade81-e1f8-4d60-9232-2e1322b007c1 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:20:32.248240 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e9272e-f897-406c-be60-e4005973f882 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535c258100df40b58a5ee44b2c21d621', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:20:32.248841 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e9272e-f897-406c-be60-e4005973f882 service cinder] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:20:32.249118 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e9272e-f897-406c-be60-e4005973f882 service cinder] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:20:32.249385 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e9272e-f897-406c-be60-e4005973f882 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:20:32.258128 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e9272e-f897-406c-be60-e4005973f882 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:20:32.259625 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e9272e-f897-406c-be60-e4005973f882 service cinder] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:20:32.268842 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1359/2738] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:20:32 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:20:33.628959 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa841382-cf27-4f3b-b996-cb8e20c71398 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:20:33.629461 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa841382-cf27-4f3b-b996-cb8e20c71398 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:20:33.629561 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa841382-cf27-4f3b-b996-cb8e20c71398 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:20:33.649211 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-aa841382-cf27-4f3b-b996-cb8e20c71398 None None] MFA Rules not processed for user `c0781112ec0c440da34254add94a593a`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:20:33.650404 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-aa841382-cf27-4f3b-b996-cb8e20c71398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:20:33.657237 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1380/2739] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:20:33 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:20:42.264336 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=17 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:20:43.642835 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=92,keystone:INSERT=3 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:20:46.518126 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cecec3-f9d1-434e-bcf7-93667c0e957e None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:20:46.518577 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cecec3-f9d1-434e-bcf7-93667c0e957e None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:20:46.518698 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cecec3-f9d1-434e-bcf7-93667c0e957e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:20:46.557354 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-80cecec3-f9d1-434e-bcf7-93667c0e957e None None] MFA Rules not processed for user `e368b4e668c54e25bff80a8e8a8c3588`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:20:46.558551 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-80cecec3-f9d1-434e-bcf7-93667c0e957e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:20:46.637839 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1360/2740] 10.222.0.49 () {58 vars in 1009 bytes} [Fri Sep 9 12:20:46 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:20:46.685432 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa841382-cf27-4f3b-b996-cb8e20c71398 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:20:46.686851 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-aa841382-cf27-4f3b-b996-cb8e20c71398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:20:46.786230 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa841382-cf27-4f3b-b996-cb8e20c71398 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:20:46.791463 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548e3b0c-def3-4372-a5c5-76fd19931daf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e652ba827849b5a3fc1213d04a1b76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:20:46.792062 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548e3b0c-def3-4372-a5c5-76fd19931daf service swift] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:20:46.792360 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548e3b0c-def3-4372-a5c5-76fd19931daf service swift] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:20:46.792627 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548e3b0c-def3-4372-a5c5-76fd19931daf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:20:46.801421 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548e3b0c-def3-4372-a5c5-76fd19931daf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:20:46.803023 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548e3b0c-def3-4372-a5c5-76fd19931daf service swift] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:20:46.810801 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1381/2741] 10.222.0.49 () {60 vars in 1384 bytes} [Fri Sep 9 12:20:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 12:20:47.324438 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb1f3d2-7c22-40fc-9ed3-3c9734e83500 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:20:47.324726 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb1f3d2-7c22-40fc-9ed3-3c9734e83500 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:20:47.325001 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb1f3d2-7c22-40fc-9ed3-3c9734e83500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:20:47.344262 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-0bb1f3d2-7c22-40fc-9ed3-3c9734e83500 None None] MFA Rules not processed for user `e368b4e668c54e25bff80a8e8a8c3588`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:20:47.346151 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-0bb1f3d2-7c22-40fc-9ed3-3c9734e83500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:20:47.353151 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1361/2742] 10.222.0.49 () {58 vars in 967 bytes} [Fri Sep 9 12:20:47 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:20:47.366062 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548e3b0c-def3-4372-a5c5-76fd19931daf service swift] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:20:47.376611 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548e3b0c-def3-4372-a5c5-76fd19931daf service swift] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:20:47.381648 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a10d2b-e4c4-4ae2-9964-b7ddf8d619e2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e652ba827849b5a3fc1213d04a1b76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:20:47.382237 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a10d2b-e4c4-4ae2-9964-b7ddf8d619e2 service swift] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:20:47.382515 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a10d2b-e4c4-4ae2-9964-b7ddf8d619e2 service swift] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:20:47.382788 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a10d2b-e4c4-4ae2-9964-b7ddf8d619e2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:20:47.390561 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a10d2b-e4c4-4ae2-9964-b7ddf8d619e2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:20:47.392889 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a10d2b-e4c4-4ae2-9964-b7ddf8d619e2 service swift] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:20:47.400770 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1382/2743] 10.222.0.49 () {60 vars in 1384 bytes} [Fri Sep 9 12:20:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 12:20:47.622610 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012ff520-dcac-410c-8c27-c31d34ae3e79 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:20:47.623100 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012ff520-dcac-410c-8c27-c31d34ae3e79 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:20:47.623213 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012ff520-dcac-410c-8c27-c31d34ae3e79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:20:47.643018 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-012ff520-dcac-410c-8c27-c31d34ae3e79 None None] MFA Rules not processed for user `e368b4e668c54e25bff80a8e8a8c3588`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:20:47.644229 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-012ff520-dcac-410c-8c27-c31d34ae3e79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:20:47.651504 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1362/2744] 10.222.0.49 () {58 vars in 1009 bytes} [Fri Sep 9 12:20:47 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:20:47.663713 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a10d2b-e4c4-4ae2-9964-b7ddf8d619e2 service swift] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:20:47.674447 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a10d2b-e4c4-4ae2-9964-b7ddf8d619e2 service swift] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:20:47.679674 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec57af3d-82f7-4eaa-8558-b66814b877f9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e652ba827849b5a3fc1213d04a1b76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:20:47.680255 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec57af3d-82f7-4eaa-8558-b66814b877f9 service swift] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:20:47.680528 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec57af3d-82f7-4eaa-8558-b66814b877f9 service swift] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:20:47.680796 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec57af3d-82f7-4eaa-8558-b66814b877f9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:20:47.688803 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec57af3d-82f7-4eaa-8558-b66814b877f9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:20:47.690565 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec57af3d-82f7-4eaa-8558-b66814b877f9 service swift] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:20:47.699016 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1383/2745] 10.222.0.49 () {60 vars in 1384 bytes} [Fri Sep 9 12:20:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 12:20:48.018734 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e486dd5-c7fa-4ca7-9b31-4fdc3d38946f None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:20:48.019032 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e486dd5-c7fa-4ca7-9b31-4fdc3d38946f None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:20:48.019300 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e486dd5-c7fa-4ca7-9b31-4fdc3d38946f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:20:48.037773 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-6e486dd5-c7fa-4ca7-9b31-4fdc3d38946f None None] MFA Rules not processed for user `e368b4e668c54e25bff80a8e8a8c3588`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:20:48.038973 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-6e486dd5-c7fa-4ca7-9b31-4fdc3d38946f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:20:48.045854 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1363/2746] 10.222.0.49 () {58 vars in 967 bytes} [Fri Sep 9 12:20:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:20:48.058380 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec57af3d-82f7-4eaa-8558-b66814b877f9 service swift] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:20:48.069241 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec57af3d-82f7-4eaa-8558-b66814b877f9 service swift] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:20:48.074195 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a6ae3b-176c-41e1-a766-4b4eed4e1eed service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e652ba827849b5a3fc1213d04a1b76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:20:48.074783 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a6ae3b-176c-41e1-a766-4b4eed4e1eed service swift] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:20:48.075095 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a6ae3b-176c-41e1-a766-4b4eed4e1eed service swift] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:20:48.075367 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a6ae3b-176c-41e1-a766-4b4eed4e1eed service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:20:48.084125 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a6ae3b-176c-41e1-a766-4b4eed4e1eed service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:20:48.085709 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a6ae3b-176c-41e1-a766-4b4eed4e1eed service swift] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:20:48.093510 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1384/2747] 10.222.0.49 () {60 vars in 1384 bytes} [Fri Sep 9 12:20:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 12:20:50.181404 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e486dd5-c7fa-4ca7-9b31-4fdc3d38946f None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:20:50.270164 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e486dd5-c7fa-4ca7-9b31-4fdc3d38946f None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:20:50.274761 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce42ec56-6e8e-4caa-8502-fcd3a0ba403f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:20:50.275417 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce42ec56-6e8e-4caa-8502-fcd3a0ba403f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:20:50.275694 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce42ec56-6e8e-4caa-8502-fcd3a0ba403f admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:20:50.275965 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce42ec56-6e8e-4caa-8502-fcd3a0ba403f admin admin] PATH_INFO: `/v3/users/c0781112ec0c440da34254add94a593a` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:20:50.277095 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce42ec56-6e8e-4caa-8502-fcd3a0ba403f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0781112ec0c440da34254add94a593a)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:20:50.279041 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce42ec56-6e8e-4caa-8502-fcd3a0ba403f admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:20:50.307878 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:20:50.327259 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-ce42ec56-6e8e-4caa-8502-fcd3a0ba403f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0781112ec0c440da34254add94a593a'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:20:50.339385 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-ce42ec56-6e8e-4caa-8502-fcd3a0ba403f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0781112ec0c440da34254add94a593a'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:20:50.346127 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-ce42ec56-6e8e-4caa-8502-fcd3a0ba403f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0781112ec0c440da34254add94a593a'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:20:50.353535 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-ce42ec56-6e8e-4caa-8502-fcd3a0ba403f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0781112ec0c440da34254add94a593a'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:20:50.354154 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-ce42ec56-6e8e-4caa-8502-fcd3a0ba403f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0781112ec0c440da34254add94a593a'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:20:50.360571 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1364/2748] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 9 12:20:50 2022] DELETE /identity/v3/users/c0781112ec0c440da34254add94a593a => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:20:50.365185 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c6556b-ea5f-4695-81ba-1e093c19585a None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:20:50.365672 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c6556b-ea5f-4695-81ba-1e093c19585a None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:20:50.365792 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c6556b-ea5f-4695-81ba-1e093c19585a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:20:50.405991 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-75c6556b-ea5f-4695-81ba-1e093c19585a None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:20:50.407249 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-75c6556b-ea5f-4695-81ba-1e093c19585a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:20:50.457655 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=100 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:20:50.465750 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1385/2749] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:20:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:20:50.475580 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce42ec56-6e8e-4caa-8502-fcd3a0ba403f admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:20:50.477007 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-ce42ec56-6e8e-4caa-8502-fcd3a0ba403f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:20:50.534047 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce42ec56-6e8e-4caa-8502-fcd3a0ba403f admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:20:50.537748 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36c59bf-0fe4-4c82-932d-019803fb9c14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:20:50.538343 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36c59bf-0fe4-4c82-932d-019803fb9c14 service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:20:50.538617 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36c59bf-0fe4-4c82-932d-019803fb9c14 service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:20:50.538886 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36c59bf-0fe4-4c82-932d-019803fb9c14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:20:50.548196 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36c59bf-0fe4-4c82-932d-019803fb9c14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:20:50.549869 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36c59bf-0fe4-4c82-932d-019803fb9c14 service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:20:50.559153 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1365/2750] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:20:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:20:50.725476 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c6556b-ea5f-4695-81ba-1e093c19585a None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:20:50.727086 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-75c6556b-ea5f-4695-81ba-1e093c19585a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:20:50.743867 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c6556b-ea5f-4695-81ba-1e093c19585a None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:20:50.748883 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4b73e2-ea24-4a8d-8e51-40328ba40835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:20:50.749515 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4b73e2-ea24-4a8d-8e51-40328ba40835 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:20:50.749914 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4b73e2-ea24-4a8d-8e51-40328ba40835 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:20:50.750198 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4b73e2-ea24-4a8d-8e51-40328ba40835 admin admin] PATH_INFO: `/v3/projects/192b3c1dcb7a4d9c952420130039ea6c` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:20:50.752560 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4b73e2-ea24-4a8d-8e51-40328ba40835 admin admin] RBAC: Authorizing `identity:delete_project(project_id=192b3c1dcb7a4d9c952420130039ea6c)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:20:50.754490 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4b73e2-ea24-4a8d-8e51-40328ba40835 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:20:50.802846 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-3e4b73e2-ea24-4a8d-8e51-40328ba40835 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '192b3c1dcb7a4d9c952420130039ea6c', 'request_id': 'req-3e4b73e2-ea24-4a8d-8e51-40328ba40835'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:20:50.808281 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-3e4b73e2-ea24-4a8d-8e51-40328ba40835 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '192b3c1dcb7a4d9c952420130039ea6c', 'request_id': 'req-3e4b73e2-ea24-4a8d-8e51-40328ba40835'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:20:50.814838 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-3e4b73e2-ea24-4a8d-8e51-40328ba40835 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '192b3c1dcb7a4d9c952420130039ea6c', 'request_id': 'req-3e4b73e2-ea24-4a8d-8e51-40328ba40835'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:20:50.826871 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-3e4b73e2-ea24-4a8d-8e51-40328ba40835 admin admin] The token cache is being invalidate because project 192b3c1dcb7a4d9c952420130039ea6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81591) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:20:50.827178 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-3e4b73e2-ea24-4a8d-8e51-40328ba40835 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:20:50.827784 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-3e4b73e2-ea24-4a8d-8e51-40328ba40835 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:20:50.829472 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1386/2751] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 9 12:20:50 2022] DELETE /identity/v3/projects/192b3c1dcb7a4d9c952420130039ea6c => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:20:50.834692 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c48db04-bc1a-4493-870f-8d56bc59c4e8 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:20:50.835021 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c48db04-bc1a-4493-870f-8d56bc59c4e8 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:20:50.835281 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c48db04-bc1a-4493-870f-8d56bc59c4e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:20:50.883416 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-6c48db04-bc1a-4493-870f-8d56bc59c4e8 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:20:50.884620 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-6c48db04-bc1a-4493-870f-8d56bc59c4e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:20:50.943926 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1366/2752] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:20:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:20:50.951949 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d64b3fe-66f6-4202-a873-bb6f99925659 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:20:50.952417 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d64b3fe-66f6-4202-a873-bb6f99925659 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:20:50.952699 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d64b3fe-66f6-4202-a873-bb6f99925659 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:20:50.972680 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-2d64b3fe-66f6-4202-a873-bb6f99925659 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:20:50.973818 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-2d64b3fe-66f6-4202-a873-bb6f99925659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:20:50.981548 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1387/2753] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:20:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:20:50.986455 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c48db04-bc1a-4493-870f-8d56bc59c4e8 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:20:50.996116 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c48db04-bc1a-4493-870f-8d56bc59c4e8 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:20:51.000982 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77a5753-20b8-409c-b187-e0dc0de135f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:20:51.001567 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77a5753-20b8-409c-b187-e0dc0de135f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:20:51.001843 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77a5753-20b8-409c-b187-e0dc0de135f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:20:51.002110 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77a5753-20b8-409c-b187-e0dc0de135f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:20:51.002787 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77a5753-20b8-409c-b187-e0dc0de135f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:20:51.003114 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77a5753-20b8-409c-b187-e0dc0de135f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:20:51.004693 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77a5753-20b8-409c-b187-e0dc0de135f1 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:20:51.015782 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1367/2754] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 9 12:20:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:20:51.020001 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d64b3fe-66f6-4202-a873-bb6f99925659 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:20:51.029121 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d64b3fe-66f6-4202-a873-bb6f99925659 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:20:51.034367 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540954c8-dea6-4a6f-88b7-5dd85ad995ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:20:51.034999 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540954c8-dea6-4a6f-88b7-5dd85ad995ee admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:20:51.035287 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540954c8-dea6-4a6f-88b7-5dd85ad995ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:20:51.035560 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540954c8-dea6-4a6f-88b7-5dd85ad995ee admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:20:51.036180 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540954c8-dea6-4a6f-88b7-5dd85ad995ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:20:51.038059 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540954c8-dea6-4a6f-88b7-5dd85ad995ee admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:20:51.068236 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1388/2755] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 9 12:20:51 2022] POST /identity/v3/projects => generated 377 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:20:51.072529 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77a5753-20b8-409c-b187-e0dc0de135f1 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:20:51.082450 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77a5753-20b8-409c-b187-e0dc0de135f1 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:20:51.087057 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:20:51.140028 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c905bc03-77a4-4062-b34f-6a5870d02401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:20:51.140628 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c905bc03-77a4-4062-b34f-6a5870d02401 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:20:51.140907 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c905bc03-77a4-4062-b34f-6a5870d02401 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:20:51.141176 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c905bc03-77a4-4062-b34f-6a5870d02401 admin admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:20:51.142710 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c905bc03-77a4-4062-b34f-6a5870d02401 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:20:51.144704 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c905bc03-77a4-4062-b34f-6a5870d02401 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:20:51.166927 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1368/2756] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:20:51 2022] POST /identity/v3/users => generated 289 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:20:51.171073 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540954c8-dea6-4a6f-88b7-5dd85ad995ee admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:20:51.181544 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540954c8-dea6-4a6f-88b7-5dd85ad995ee admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:20:51.186105 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d756c1-7e50-4df0-9dc9-637f97c006bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:20:51.186694 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d756c1-7e50-4df0-9dc9-637f97c006bd admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:20:51.187010 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d756c1-7e50-4df0-9dc9-637f97c006bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:20:51.187282 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d756c1-7e50-4df0-9dc9-637f97c006bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:20:51.187944 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d756c1-7e50-4df0-9dc9-637f97c006bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:20:51.189613 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d756c1-7e50-4df0-9dc9-637f97c006bd admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:20:51.199406 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1389/2757] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:20:51 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:20:51.203251 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c905bc03-77a4-4062-b34f-6a5870d02401 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:20:51.215855 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c905bc03-77a4-4062-b34f-6a5870d02401 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:20:51.221028 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9939aa9b-ceb9-4129-a883-8921fa030465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:20:51.221620 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9939aa9b-ceb9-4129-a883-8921fa030465 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:20:51.221896 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9939aa9b-ceb9-4129-a883-8921fa030465 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:20:51.222163 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9939aa9b-ceb9-4129-a883-8921fa030465 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:20:51.222868 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9939aa9b-ceb9-4129-a883-8921fa030465 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:20:51.224436 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9939aa9b-ceb9-4129-a883-8921fa030465 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:20:51.234863 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1369/2758] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:20:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:20:51.239349 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d756c1-7e50-4df0-9dc9-637f97c006bd admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:20:51.249066 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d756c1-7e50-4df0-9dc9-637f97c006bd admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:20:51.253735 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e756b7-d0c2-406f-9bdb-5ec33acaab22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:20:51.254377 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e756b7-d0c2-406f-9bdb-5ec33acaab22 admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:20:51.254649 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e756b7-d0c2-406f-9bdb-5ec33acaab22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:20:51.254947 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e756b7-d0c2-406f-9bdb-5ec33acaab22 admin admin] PATH_INFO: `/v3/projects/c9c41d17597b4d3a8ffafa3d5bb87916/users/e2391d64c9304317bfb53e79ee0ca764/roles/eecd2f160dd844f5906f3d345780c501` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:20:51.269560 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e756b7-d0c2-406f-9bdb-5ec33acaab22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9c41d17597b4d3a8ffafa3d5bb87916, user_id=e2391d64c9304317bfb53e79ee0ca764, role_id=eecd2f160dd844f5906f3d345780c501)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:20:51.272234 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e756b7-d0c2-406f-9bdb-5ec33acaab22 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:20:51.285697 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1390/2759] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:20:51 2022] PUT /identity/v3/projects/c9c41d17597b4d3a8ffafa3d5bb87916/users/e2391d64c9304317bfb53e79ee0ca764/roles/eecd2f160dd844f5906f3d345780c501 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:20:51.290775 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5144706c-a5da-4a89-bdbe-ead0dbe94527 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:20:51.291091 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5144706c-a5da-4a89-bdbe-ead0dbe94527 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:20:51.291364 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5144706c-a5da-4a89-bdbe-ead0dbe94527 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:20:51.310251 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-5144706c-a5da-4a89-bdbe-ead0dbe94527 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:20:51.311484 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-5144706c-a5da-4a89-bdbe-ead0dbe94527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:20:51.370955 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1370/2760] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:20:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:20:51.380616 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e756b7-d0c2-406f-9bdb-5ec33acaab22 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:20:51.382300 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-67e756b7-d0c2-406f-9bdb-5ec33acaab22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:20:51.438561 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e756b7-d0c2-406f-9bdb-5ec33acaab22 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:20:51.442253 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db7116d-0884-4e7f-bd6b-ce791192ab31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:20:51.442845 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db7116d-0884-4e7f-bd6b-ce791192ab31 service neutron] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:20:51.443148 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db7116d-0884-4e7f-bd6b-ce791192ab31 service neutron] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:20:51.443413 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db7116d-0884-4e7f-bd6b-ce791192ab31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:20:51.452126 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db7116d-0884-4e7f-bd6b-ce791192ab31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:20:51.453531 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db7116d-0884-4e7f-bd6b-ce791192ab31 service neutron] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:20:51.463593 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1391/2761] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:20:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:20:55.415497 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acda4bf-4d7f-4938-b383-fdfa3a291741 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:20:55.416430 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acda4bf-4d7f-4938-b383-fdfa3a291741 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:20:55.416809 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acda4bf-4d7f-4938-b383-fdfa3a291741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:20:55.446528 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-5acda4bf-4d7f-4938-b383-fdfa3a291741 None None] MFA Rules not processed for user `e2391d64c9304317bfb53e79ee0ca764`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:20:55.447716 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-5acda4bf-4d7f-4938-b383-fdfa3a291741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:20:55.513601 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1371/2762] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:20:55 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:20:55.517624 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db7116d-0884-4e7f-bd6b-ce791192ab31 service neutron] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:20:55.536346 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db7116d-0884-4e7f-bd6b-ce791192ab31 service neutron] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:20:55.541139 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4e214e-6b92-4ee3-9792-98b9450c527e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:20:55.541747 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4e214e-6b92-4ee3-9792-98b9450c527e admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:20:55.542086 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4e214e-6b92-4ee3-9792-98b9450c527e admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:20:55.542355 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4e214e-6b92-4ee3-9792-98b9450c527e admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:20:55.544414 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4e214e-6b92-4ee3-9792-98b9450c527e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:20:55.546509 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4e214e-6b92-4ee3-9792-98b9450c527e admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:20:55.570056 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:20:55.583632 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acda4bf-4d7f-4938-b383-fdfa3a291741 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:20:55.583984 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1392/2763] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 9 12:20:55 2022] POST /identity/v3/projects => generated 377 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:20:55.595815 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acda4bf-4d7f-4938-b383-fdfa3a291741 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:20:55.610674 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:20:55.667502 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167fde89-dee4-4a9e-9773-171793194e2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:20:55.668148 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167fde89-dee4-4a9e-9773-171793194e2e admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:20:55.668432 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167fde89-dee4-4a9e-9773-171793194e2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:20:55.668704 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167fde89-dee4-4a9e-9773-171793194e2e admin admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:20:55.669341 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167fde89-dee4-4a9e-9773-171793194e2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:20:55.671355 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167fde89-dee4-4a9e-9773-171793194e2e admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:20:55.695059 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1372/2764] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:20:55 2022] POST /identity/v3/users => generated 295 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:20:55.697697 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4e214e-6b92-4ee3-9792-98b9450c527e admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:20:55.709893 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4e214e-6b92-4ee3-9792-98b9450c527e admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:20:55.714634 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95740058-6777-46cc-949c-43053d897789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:20:55.715312 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95740058-6777-46cc-949c-43053d897789 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:20:55.715592 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95740058-6777-46cc-949c-43053d897789 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:20:55.716077 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95740058-6777-46cc-949c-43053d897789 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:20:55.716737 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95740058-6777-46cc-949c-43053d897789 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:20:55.718336 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95740058-6777-46cc-949c-43053d897789 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:20:55.730455 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1393/2765] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:20:55 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:20:55.735032 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167fde89-dee4-4a9e-9773-171793194e2e admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:20:55.745270 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167fde89-dee4-4a9e-9773-171793194e2e admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:20:55.750044 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e93cc3e-1cd9-43be-baff-ce253b76fa5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:20:55.750773 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e93cc3e-1cd9-43be-baff-ce253b76fa5e admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:20:55.751098 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e93cc3e-1cd9-43be-baff-ce253b76fa5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:20:55.751369 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e93cc3e-1cd9-43be-baff-ce253b76fa5e admin admin] PATH_INFO: `/v3/projects/99c91073ba464c738a6055c59d2d9ea6/users/e7d1f7afdd42442484c3e8d93b12878c/roles/3f2ca33902a04b6dbc2ef7d06d7289d3` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:20:55.768096 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e93cc3e-1cd9-43be-baff-ce253b76fa5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=99c91073ba464c738a6055c59d2d9ea6, user_id=e7d1f7afdd42442484c3e8d93b12878c, role_id=3f2ca33902a04b6dbc2ef7d06d7289d3)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:20:55.770762 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e93cc3e-1cd9-43be-baff-ce253b76fa5e admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:20:55.785906 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1373/2766] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:20:55 2022] PUT /identity/v3/projects/99c91073ba464c738a6055c59d2d9ea6/users/e7d1f7afdd42442484c3e8d93b12878c/roles/3f2ca33902a04b6dbc2ef7d06d7289d3 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:20:55.789980 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95740058-6777-46cc-949c-43053d897789 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:20:55.801074 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95740058-6777-46cc-949c-43053d897789 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:20:55.861840 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996350f7-af41-42d4-81aa-761a9d7a55f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:20:55.862459 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996350f7-af41-42d4-81aa-761a9d7a55f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:20:55.862730 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996350f7-af41-42d4-81aa-761a9d7a55f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:20:55.863027 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996350f7-af41-42d4-81aa-761a9d7a55f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:20:55.863666 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996350f7-af41-42d4-81aa-761a9d7a55f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:20:55.865252 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996350f7-af41-42d4-81aa-761a9d7a55f1 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:20:55.876330 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1394/2767] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:20:55 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:20:55.880388 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e93cc3e-1cd9-43be-baff-ce253b76fa5e admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:20:55.890758 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e93cc3e-1cd9-43be-baff-ce253b76fa5e admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:20:55.896142 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df5624a-dcad-4de1-88ce-61e23397469b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:20:55.896807 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df5624a-dcad-4de1-88ce-61e23397469b admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:20:55.897141 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df5624a-dcad-4de1-88ce-61e23397469b admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:20:55.897417 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df5624a-dcad-4de1-88ce-61e23397469b admin admin] PATH_INFO: `/v3/projects/99c91073ba464c738a6055c59d2d9ea6/users/e7d1f7afdd42442484c3e8d93b12878c/roles/3f2ca33902a04b6dbc2ef7d06d7289d3` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:20:55.899407 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df5624a-dcad-4de1-88ce-61e23397469b admin admin] RBAC: Authorizing `identity:create_grant(project_id=99c91073ba464c738a6055c59d2d9ea6, user_id=e7d1f7afdd42442484c3e8d93b12878c, role_id=3f2ca33902a04b6dbc2ef7d06d7289d3)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:20:55.902051 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df5624a-dcad-4de1-88ce-61e23397469b admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:20:55.913895 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1374/2768] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:20:55 2022] PUT /identity/v3/projects/99c91073ba464c738a6055c59d2d9ea6/users/e7d1f7afdd42442484c3e8d93b12878c/roles/3f2ca33902a04b6dbc2ef7d06d7289d3 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:21:00.277827 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcd6a86-2d48-4251-bc21-07efd1e9e10a None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:21:00.278297 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcd6a86-2d48-4251-bc21-07efd1e9e10a None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:21:00.278394 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcd6a86-2d48-4251-bc21-07efd1e9e10a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:21:00.307783 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-3fcd6a86-2d48-4251-bc21-07efd1e9e10a None None] MFA Rules not processed for user `e7d1f7afdd42442484c3e8d93b12878c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:21:00.308950 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-3fcd6a86-2d48-4251-bc21-07efd1e9e10a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:21:00.382366 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1395/2769] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:21:00 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:21:00.398452 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df5624a-dcad-4de1-88ce-61e23397469b admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:21:00.400399 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-4df5624a-dcad-4de1-88ce-61e23397469b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:21:00.502147 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df5624a-dcad-4de1-88ce-61e23397469b admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:21:00.507726 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b514e0-917d-4d14-bf3f-c81beb91f610 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70bac169d5cd43e1926f310344ac11fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:21:00.508332 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b514e0-917d-4d14-bf3f-c81beb91f610 service nova] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:21:00.508616 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b514e0-917d-4d14-bf3f-c81beb91f610 service nova] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:21:00.508886 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b514e0-917d-4d14-bf3f-c81beb91f610 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:21:00.563319 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:21:00.569113 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b514e0-917d-4d14-bf3f-c81beb91f610 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:21:00.571028 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b514e0-917d-4d14-bf3f-c81beb91f610 service nova] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:21:00.580587 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1375/2770] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:21:00 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:21:10.379517 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=68 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:21:10.576159 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=6 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:21:17.022248 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c15c3dd-e98d-4c1a-a08e-07c9f3e9a635 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:21:17.022720 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c15c3dd-e98d-4c1a-a08e-07c9f3e9a635 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:21:17.022822 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c15c3dd-e98d-4c1a-a08e-07c9f3e9a635 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:21:17.041804 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-9c15c3dd-e98d-4c1a-a08e-07c9f3e9a635 None None] MFA Rules not processed for user `e2391d64c9304317bfb53e79ee0ca764`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:21:17.043036 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-9c15c3dd-e98d-4c1a-a08e-07c9f3e9a635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:21:17.050848 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1396/2771] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:21:17 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:21:19.336532 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b514e0-917d-4d14-bf3f-c81beb91f610 service nova] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:21:19.338206 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-46b514e0-917d-4d14-bf3f-c81beb91f610 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:21:19.429212 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b514e0-917d-4d14-bf3f-c81beb91f610 service nova] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:21:19.434306 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cade31e8-e4ed-4780-a75b-5abc7007885a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3adb2f0583c4ff782049dacdb3ee556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:21:19.434921 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cade31e8-e4ed-4780-a75b-5abc7007885a service placement] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:21:19.435223 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cade31e8-e4ed-4780-a75b-5abc7007885a service placement] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:21:19.435497 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cade31e8-e4ed-4780-a75b-5abc7007885a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:21:19.437778 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-cade31e8-e4ed-4780-a75b-5abc7007885a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:21:19.449089 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cade31e8-e4ed-4780-a75b-5abc7007885a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:21:19.450650 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cade31e8-e4ed-4780-a75b-5abc7007885a service placement] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:21:19.461078 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1376/2772] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:21:19 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:21:27.036474 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=3 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:21:29.456829 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=41 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:21:34.154517 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c15c3dd-e98d-4c1a-a08e-07c9f3e9a635 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:21:34.220852 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c15c3dd-e98d-4c1a-a08e-07c9f3e9a635 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:21:34.224367 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3918d8-25f2-48de-8755-0e555dcd19bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:21:34.224966 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3918d8-25f2-48de-8755-0e555dcd19bd service neutron] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:21:34.225241 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3918d8-25f2-48de-8755-0e555dcd19bd service neutron] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:21:34.225507 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3918d8-25f2-48de-8755-0e555dcd19bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:21:34.234377 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3918d8-25f2-48de-8755-0e555dcd19bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:21:34.235817 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3918d8-25f2-48de-8755-0e555dcd19bd service neutron] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:21:34.244961 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1397/2773] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:21:34 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:21:44.240849 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=30 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:22:59.406481 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cade31e8-e4ed-4780-a75b-5abc7007885a service placement] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:22:59.415839 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cade31e8-e4ed-4780-a75b-5abc7007885a service placement] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:22:59.419440 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17928178-7b86-4a32-8ef5-646b820cbc1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:22:59.420180 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17928178-7b86-4a32-8ef5-646b820cbc1b service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:22:59.420458 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17928178-7b86-4a32-8ef5-646b820cbc1b service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:22:59.420991 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17928178-7b86-4a32-8ef5-646b820cbc1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:22:59.422694 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-17928178-7b86-4a32-8ef5-646b820cbc1b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:22:59.436065 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17928178-7b86-4a32-8ef5-646b820cbc1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:22:59.437462 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17928178-7b86-4a32-8ef5-646b820cbc1b service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:22:59.448119 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1377/2774] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:22:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:23:00.631134 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3918d8-25f2-48de-8755-0e555dcd19bd service neutron] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:23:00.639970 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3918d8-25f2-48de-8755-0e555dcd19bd service neutron] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:23:00.643603 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f04260-e519-4e4f-a3c1-3ce5670c9c5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:23:00.644264 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f04260-e519-4e4f-a3c1-3ce5670c9c5d service neutron] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:23:00.644604 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f04260-e519-4e4f-a3c1-3ce5670c9c5d service neutron] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:23:00.644939 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f04260-e519-4e4f-a3c1-3ce5670c9c5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:23:00.646692 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-81f04260-e519-4e4f-a3c1-3ce5670c9c5d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:23:00.660735 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f04260-e519-4e4f-a3c1-3ce5670c9c5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:23:00.662117 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f04260-e519-4e4f-a3c1-3ce5670c9c5d service neutron] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:23:00.673819 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1398/2775] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:23:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:23:05.859003 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b32436-c3d7-4a53-a55f-5c59754953cb None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:23:05.859550 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b32436-c3d7-4a53-a55f-5c59754953cb None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:23:05.859675 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b32436-c3d7-4a53-a55f-5c59754953cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:23:05.878111 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-e2b32436-c3d7-4a53-a55f-5c59754953cb None None] MFA Rules not processed for user `870f5366f958496d93c51e4b573e2487`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:23:05.879329 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-e2b32436-c3d7-4a53-a55f-5c59754953cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:23:05.933424 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1378/2776] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:23:05 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:23:10.667861 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=6 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:23:15.926538 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=31 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:23:16.858457 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f04260-e519-4e4f-a3c1-3ce5670c9c5d service neutron] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:23:16.859998 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-81f04260-e519-4e4f-a3c1-3ce5670c9c5d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:23:16.883741 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f04260-e519-4e4f-a3c1-3ce5670c9c5d service neutron] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:23:16.888228 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd0fa8c-6845-4c37-953a-089c75955a0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:23:16.888829 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd0fa8c-6845-4c37-953a-089c75955a0a service neutron] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:23:16.889098 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd0fa8c-6845-4c37-953a-089c75955a0a service neutron] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:23:16.889360 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd0fa8c-6845-4c37-953a-089c75955a0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:23:16.891076 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-dfd0fa8c-6845-4c37-953a-089c75955a0a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:23:16.900786 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd0fa8c-6845-4c37-953a-089c75955a0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:23:16.902185 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd0fa8c-6845-4c37-953a-089c75955a0a service neutron] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:23:16.910809 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1399/2777] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:23:16 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:23:26.907128 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=11 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:23:34.269807 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b32436-c3d7-4a53-a55f-5c59754953cb None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:23:34.271596 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-e2b32436-c3d7-4a53-a55f-5c59754953cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:23:34.289538 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b32436-c3d7-4a53-a55f-5c59754953cb None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:23:34.296750 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f2744e-65b2-4a9b-87b8-55cdcd922149 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70bac169d5cd43e1926f310344ac11fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:23:34.297563 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f2744e-65b2-4a9b-87b8-55cdcd922149 service nova] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:23:34.297997 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f2744e-65b2-4a9b-87b8-55cdcd922149 service nova] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:23:34.298390 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f2744e-65b2-4a9b-87b8-55cdcd922149 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:23:34.300253 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-b1f2744e-65b2-4a9b-87b8-55cdcd922149 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:23:34.315715 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f2744e-65b2-4a9b-87b8-55cdcd922149 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:23:34.320793 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f2744e-65b2-4a9b-87b8-55cdcd922149 service nova] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:23:34.333462 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1379/2778] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:23:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:23:35.654858 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909b4c5a-73c4-478c-b00d-f3aff261439a None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:23:35.655397 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909b4c5a-73c4-478c-b00d-f3aff261439a None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:23:35.655397 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909b4c5a-73c4-478c-b00d-f3aff261439a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:23:35.674944 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-909b4c5a-73c4-478c-b00d-f3aff261439a None None] MFA Rules not processed for user `e2391d64c9304317bfb53e79ee0ca764`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:23:35.676142 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-909b4c5a-73c4-478c-b00d-f3aff261439a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:23:35.683054 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1400/2779] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:23:35 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:23:35.999337 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f2744e-65b2-4a9b-87b8-55cdcd922149 service nova] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:23:36.008379 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f2744e-65b2-4a9b-87b8-55cdcd922149 service nova] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:23:36.012431 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96bcdb4-3291-43ac-930e-14e73c2161d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:23:36.013007 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96bcdb4-3291-43ac-930e-14e73c2161d1 service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:23:36.013283 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96bcdb4-3291-43ac-930e-14e73c2161d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:23:36.013548 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96bcdb4-3291-43ac-930e-14e73c2161d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:23:36.015256 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-e96bcdb4-3291-43ac-930e-14e73c2161d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:23:36.024397 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96bcdb4-3291-43ac-930e-14e73c2161d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:23:36.025791 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96bcdb4-3291-43ac-930e-14e73c2161d1 service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:23:36.034107 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1380/2780] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:23:35 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:23:36.481326 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909b4c5a-73c4-478c-b00d-f3aff261439a None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:23:36.483126 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-909b4c5a-73c4-478c-b00d-f3aff261439a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:23:36.498650 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909b4c5a-73c4-478c-b00d-f3aff261439a None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:23:36.503713 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4147ae-956d-442b-ae12-5d55370337d9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3adb2f0583c4ff782049dacdb3ee556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:23:36.504320 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4147ae-956d-442b-ae12-5d55370337d9 service placement] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:23:36.504601 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4147ae-956d-442b-ae12-5d55370337d9 service placement] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:23:36.504872 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4147ae-956d-442b-ae12-5d55370337d9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:23:36.506596 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-df4147ae-956d-442b-ae12-5d55370337d9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:23:36.519098 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4147ae-956d-442b-ae12-5d55370337d9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:23:36.520675 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4147ae-956d-442b-ae12-5d55370337d9 service placement] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:23:36.531106 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1401/2781] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:23:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:23:46.030628 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=12 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:23:46.526595 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=9 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:23:48.141343 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96bcdb4-3291-43ac-930e-14e73c2161d1 service neutron] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:23:48.158302 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96bcdb4-3291-43ac-930e-14e73c2161d1 service neutron] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:23:48.163845 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7de27f2-c1ba-4d9a-9aca-361f8bf051c7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70bac169d5cd43e1926f310344ac11fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:23:48.164447 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7de27f2-c1ba-4d9a-9aca-361f8bf051c7 service nova] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:23:48.164724 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7de27f2-c1ba-4d9a-9aca-361f8bf051c7 service nova] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:23:48.164992 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7de27f2-c1ba-4d9a-9aca-361f8bf051c7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:23:48.166750 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-e7de27f2-c1ba-4d9a-9aca-361f8bf051c7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:23:48.181514 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7de27f2-c1ba-4d9a-9aca-361f8bf051c7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:23:48.183685 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7de27f2-c1ba-4d9a-9aca-361f8bf051c7 service nova] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:23:48.197468 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1381/2782] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:23:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:23:52.342169 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4147ae-956d-442b-ae12-5d55370337d9 service placement] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:23:52.354188 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4147ae-956d-442b-ae12-5d55370337d9 service placement] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:23:52.360378 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b6e67a-7ce4-4bb2-a5e8-7d32a17b368d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70bac169d5cd43e1926f310344ac11fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:23:52.360987 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b6e67a-7ce4-4bb2-a5e8-7d32a17b368d service nova] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:23:52.361260 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b6e67a-7ce4-4bb2-a5e8-7d32a17b368d service nova] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:23:52.361524 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b6e67a-7ce4-4bb2-a5e8-7d32a17b368d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:23:52.363286 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-b9b6e67a-7ce4-4bb2-a5e8-7d32a17b368d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:23:52.375987 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b6e67a-7ce4-4bb2-a5e8-7d32a17b368d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:23:52.377561 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b6e67a-7ce4-4bb2-a5e8-7d32a17b368d service nova] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:23:52.388887 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1402/2783] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:23:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:23:58.191629 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=6 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:24:02.383511 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=6 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:24:23.131555 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7de27f2-c1ba-4d9a-9aca-361f8bf051c7 service nova] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:23.147281 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7de27f2-c1ba-4d9a-9aca-361f8bf051c7 service nova] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:23.157724 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766269ce-820c-4694-89fc-7be64342c423 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3adb2f0583c4ff782049dacdb3ee556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:23.158610 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766269ce-820c-4694-89fc-7be64342c423 service placement] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:23.159108 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766269ce-820c-4694-89fc-7be64342c423 service placement] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:23.159551 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766269ce-820c-4694-89fc-7be64342c423 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:23.162445 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-766269ce-820c-4694-89fc-7be64342c423 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:24:23.179996 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766269ce-820c-4694-89fc-7be64342c423 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:23.182628 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766269ce-820c-4694-89fc-7be64342c423 service placement] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:23.199009 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1382/2784] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:24:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:24:23.215325 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b6e67a-7ce4-4bb2-a5e8-7d32a17b368d service nova] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:23.217788 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-b9b6e67a-7ce4-4bb2-a5e8-7d32a17b368d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:24:23.305658 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b6e67a-7ce4-4bb2-a5e8-7d32a17b368d service nova] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:23.311734 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e9ac45-da32-46ca-9bd9-dc2bfb1ffcf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:23.312440 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e9ac45-da32-46ca-9bd9-dc2bfb1ffcf3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:23.312793 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e9ac45-da32-46ca-9bd9-dc2bfb1ffcf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:23.313064 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e9ac45-da32-46ca-9bd9-dc2bfb1ffcf3 admin admin] PATH_INFO: `/v3/users/870f5366f958496d93c51e4b573e2487` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:23.314194 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e9ac45-da32-46ca-9bd9-dc2bfb1ffcf3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=870f5366f958496d93c51e4b573e2487)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:23.316139 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e9ac45-da32-46ca-9bd9-dc2bfb1ffcf3 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:23.363881 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-c5e9ac45-da32-46ca-9bd9-dc2bfb1ffcf3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '870f5366f958496d93c51e4b573e2487'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:23.370766 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-c5e9ac45-da32-46ca-9bd9-dc2bfb1ffcf3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '870f5366f958496d93c51e4b573e2487'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:23.383032 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-c5e9ac45-da32-46ca-9bd9-dc2bfb1ffcf3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '870f5366f958496d93c51e4b573e2487'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:23.390323 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-c5e9ac45-da32-46ca-9bd9-dc2bfb1ffcf3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '870f5366f958496d93c51e4b573e2487'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:23.390955 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-c5e9ac45-da32-46ca-9bd9-dc2bfb1ffcf3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '870f5366f958496d93c51e4b573e2487'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:23.396883 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1403/2785] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 9 12:24:23 2022] DELETE /identity/v3/users/870f5366f958496d93c51e4b573e2487 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:24:23.572137 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766269ce-820c-4694-89fc-7be64342c423 service placement] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:23.588469 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766269ce-820c-4694-89fc-7be64342c423 service placement] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:23.649819 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f722d83-2ea3-4e4e-b0ee-a2079e37c1b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:23.650506 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f722d83-2ea3-4e4e-b0ee-a2079e37c1b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:23.650727 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f722d83-2ea3-4e4e-b0ee-a2079e37c1b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:23.651030 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f722d83-2ea3-4e4e-b0ee-a2079e37c1b3 admin admin] PATH_INFO: `/v3/projects/69e19a94fd744d9e94d9f5ff5b089f15` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:23.652153 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f722d83-2ea3-4e4e-b0ee-a2079e37c1b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=69e19a94fd744d9e94d9f5ff5b089f15)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:23.654786 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f722d83-2ea3-4e4e-b0ee-a2079e37c1b3 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:23.700328 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-0f722d83-2ea3-4e4e-b0ee-a2079e37c1b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69e19a94fd744d9e94d9f5ff5b089f15', 'request_id': 'req-0f722d83-2ea3-4e4e-b0ee-a2079e37c1b3'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:23.705969 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-0f722d83-2ea3-4e4e-b0ee-a2079e37c1b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69e19a94fd744d9e94d9f5ff5b089f15', 'request_id': 'req-0f722d83-2ea3-4e4e-b0ee-a2079e37c1b3'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:23.713314 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-0f722d83-2ea3-4e4e-b0ee-a2079e37c1b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69e19a94fd744d9e94d9f5ff5b089f15', 'request_id': 'req-0f722d83-2ea3-4e4e-b0ee-a2079e37c1b3'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:23.725325 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-0f722d83-2ea3-4e4e-b0ee-a2079e37c1b3 admin admin] The token cache is being invalidate because project 69e19a94fd744d9e94d9f5ff5b089f15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:24:23.725609 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-0f722d83-2ea3-4e4e-b0ee-a2079e37c1b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:23.726232 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-0f722d83-2ea3-4e4e-b0ee-a2079e37c1b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:23.727795 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1383/2786] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 9 12:24:23 2022] DELETE /identity/v3/projects/69e19a94fd744d9e94d9f5ff5b089f15 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:24:23.733325 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e61719e-da32-486c-8c22-390f5a2e958a None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:23.733608 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e61719e-da32-486c-8c22-390f5a2e958a None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:23.733879 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e61719e-da32-486c-8c22-390f5a2e958a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:23.753086 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-5e61719e-da32-486c-8c22-390f5a2e958a None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:24:23.754343 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-5e61719e-da32-486c-8c22-390f5a2e958a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:24:23.811587 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1404/2787] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:24:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:24:23.819444 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d28d79-f8c4-4a63-882c-d9b56ca43ba8 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:23.819725 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d28d79-f8c4-4a63-882c-d9b56ca43ba8 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:23.820011 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d28d79-f8c4-4a63-882c-d9b56ca43ba8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:23.839009 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-35d28d79-f8c4-4a63-882c-d9b56ca43ba8 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:24:23.840186 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-35d28d79-f8c4-4a63-882c-d9b56ca43ba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:24:23.848403 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1384/2788] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:24:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:24:23.852552 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e61719e-da32-486c-8c22-390f5a2e958a None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:23.862996 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e61719e-da32-486c-8c22-390f5a2e958a None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:23.867411 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a766f55f-2ed9-4b40-b26b-4a9049dda491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:23.867994 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a766f55f-2ed9-4b40-b26b-4a9049dda491 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:23.868273 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a766f55f-2ed9-4b40-b26b-4a9049dda491 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:23.868543 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a766f55f-2ed9-4b40-b26b-4a9049dda491 admin admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:23.869177 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a766f55f-2ed9-4b40-b26b-4a9049dda491 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:24:23.869475 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a766f55f-2ed9-4b40-b26b-4a9049dda491 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:23.871071 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a766f55f-2ed9-4b40-b26b-4a9049dda491 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:23.882416 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1405/2789] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 9 12:24:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:24:23.886485 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d28d79-f8c4-4a63-882c-d9b56ca43ba8 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:23.895943 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d28d79-f8c4-4a63-882c-d9b56ca43ba8 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:23.900437 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1d2c4c-57ec-4e50-a814-7017af512f21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:23.901077 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1d2c4c-57ec-4e50-a814-7017af512f21 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:23.901294 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1d2c4c-57ec-4e50-a814-7017af512f21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:23.901556 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1d2c4c-57ec-4e50-a814-7017af512f21 admin admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:23.902171 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1d2c4c-57ec-4e50-a814-7017af512f21 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:23.904830 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1d2c4c-57ec-4e50-a814-7017af512f21 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:23.940236 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1385/2790] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 9 12:24:23 2022] POST /identity/v3/projects => generated 385 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:24:23.944265 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a766f55f-2ed9-4b40-b26b-4a9049dda491 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:23.954167 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a766f55f-2ed9-4b40-b26b-4a9049dda491 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:23.957765 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:24:24.012620 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a084d6d1-8e63-40e5-ad74-1489922fa6fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:24.014065 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a084d6d1-8e63-40e5-ad74-1489922fa6fd admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:24.014341 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a084d6d1-8e63-40e5-ad74-1489922fa6fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:24.014609 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a084d6d1-8e63-40e5-ad74-1489922fa6fd admin admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:24.015253 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a084d6d1-8e63-40e5-ad74-1489922fa6fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:24.017362 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a084d6d1-8e63-40e5-ad74-1489922fa6fd admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:24.038492 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1406/2791] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:24:23 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:24:24.042712 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1d2c4c-57ec-4e50-a814-7017af512f21 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:24.053210 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1d2c4c-57ec-4e50-a814-7017af512f21 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:24.057755 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24d1e99-7467-4a3e-8b81-141fb6d4c60c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:24.058347 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24d1e99-7467-4a3e-8b81-141fb6d4c60c admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:24.058627 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24d1e99-7467-4a3e-8b81-141fb6d4c60c admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:24.058922 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24d1e99-7467-4a3e-8b81-141fb6d4c60c admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:24.059561 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24d1e99-7467-4a3e-8b81-141fb6d4c60c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:24.061400 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24d1e99-7467-4a3e-8b81-141fb6d4c60c admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:24.071152 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1386/2792] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:24:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:24:24.075040 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a084d6d1-8e63-40e5-ad74-1489922fa6fd admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:24.085494 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a084d6d1-8e63-40e5-ad74-1489922fa6fd admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:24.089995 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f90f705-ff28-4be5-b903-b79c2d935612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:24.090570 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f90f705-ff28-4be5-b903-b79c2d935612 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:24.090846 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f90f705-ff28-4be5-b903-b79c2d935612 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:24.091759 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f90f705-ff28-4be5-b903-b79c2d935612 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:24.092630 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f90f705-ff28-4be5-b903-b79c2d935612 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:24.094197 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f90f705-ff28-4be5-b903-b79c2d935612 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:24.104834 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1407/2793] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:24:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:24:24.108906 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24d1e99-7467-4a3e-8b81-141fb6d4c60c admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:24.118864 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24d1e99-7467-4a3e-8b81-141fb6d4c60c admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:24.123880 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c17d36f-1c3c-4b9e-8dcd-135aeb3ccf1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:24.124519 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c17d36f-1c3c-4b9e-8dcd-135aeb3ccf1e admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:24.124798 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c17d36f-1c3c-4b9e-8dcd-135aeb3ccf1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:24.125073 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c17d36f-1c3c-4b9e-8dcd-135aeb3ccf1e admin admin] PATH_INFO: `/v3/projects/25eab5f0def340079cec8987940c49c3/users/5ce9db5f04634742a34d99a3332831c3/roles/eecd2f160dd844f5906f3d345780c501` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:24.140533 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c17d36f-1c3c-4b9e-8dcd-135aeb3ccf1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=25eab5f0def340079cec8987940c49c3, user_id=5ce9db5f04634742a34d99a3332831c3, role_id=eecd2f160dd844f5906f3d345780c501)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:24.143350 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c17d36f-1c3c-4b9e-8dcd-135aeb3ccf1e admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:24.157247 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1387/2794] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:24:24 2022] PUT /identity/v3/projects/25eab5f0def340079cec8987940c49c3/users/5ce9db5f04634742a34d99a3332831c3/roles/eecd2f160dd844f5906f3d345780c501 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:24:24.168686 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36942d00-0200-4b2a-9d7c-929594d4fde1 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:24.169000 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36942d00-0200-4b2a-9d7c-929594d4fde1 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:24.169284 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36942d00-0200-4b2a-9d7c-929594d4fde1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:24.197814 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-36942d00-0200-4b2a-9d7c-929594d4fde1 None None] MFA Rules not processed for user `5ce9db5f04634742a34d99a3332831c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:24:24.199035 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-36942d00-0200-4b2a-9d7c-929594d4fde1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:24:24.264476 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1408/2795] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:24:24 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:24:24.268959 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c17d36f-1c3c-4b9e-8dcd-135aeb3ccf1e admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:24.274608 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:24:24.331993 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c17d36f-1c3c-4b9e-8dcd-135aeb3ccf1e admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:24.336738 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab0c6a7-7c42-4e8e-b16a-711ecff253ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:24.337339 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab0c6a7-7c42-4e8e-b16a-711ecff253ce admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:24.337620 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab0c6a7-7c42-4e8e-b16a-711ecff253ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:24.337891 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab0c6a7-7c42-4e8e-b16a-711ecff253ce admin admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:24.338519 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab0c6a7-7c42-4e8e-b16a-711ecff253ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:24.340457 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab0c6a7-7c42-4e8e-b16a-711ecff253ce admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:24.366947 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1388/2796] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 9 12:24:24 2022] POST /identity/v3/projects => generated 387 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:24:24.371034 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36942d00-0200-4b2a-9d7c-929594d4fde1 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:24.382130 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36942d00-0200-4b2a-9d7c-929594d4fde1 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:24.439856 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f7652b-e25a-4bff-bb02-a2e6ad61274c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:24.440387 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f7652b-e25a-4bff-bb02-a2e6ad61274c admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:24.440692 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f7652b-e25a-4bff-bb02-a2e6ad61274c admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:24.440964 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f7652b-e25a-4bff-bb02-a2e6ad61274c admin admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:24.441595 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f7652b-e25a-4bff-bb02-a2e6ad61274c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:24.443555 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f7652b-e25a-4bff-bb02-a2e6ad61274c admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:24.465121 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1409/2797] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:24:24 2022] POST /identity/v3/users => generated 300 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:24:24.469294 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab0c6a7-7c42-4e8e-b16a-711ecff253ce admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:24.479513 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab0c6a7-7c42-4e8e-b16a-711ecff253ce admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:24.484302 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ade7c01-bf59-4080-9ad9-f07ad464cbe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:24.484896 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ade7c01-bf59-4080-9ad9-f07ad464cbe9 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:24.485174 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ade7c01-bf59-4080-9ad9-f07ad464cbe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:24.485446 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ade7c01-bf59-4080-9ad9-f07ad464cbe9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:24.486077 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ade7c01-bf59-4080-9ad9-f07ad464cbe9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:24.487707 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ade7c01-bf59-4080-9ad9-f07ad464cbe9 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:24.499805 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1389/2798] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:24:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:24:24.503891 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f7652b-e25a-4bff-bb02-a2e6ad61274c admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:24.510160 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:24:24.515308 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f7652b-e25a-4bff-bb02-a2e6ad61274c admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:24.521037 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90f1b8a-818a-4ef9-ae0e-4ac30497fc13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:24.521679 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90f1b8a-818a-4ef9-ae0e-4ac30497fc13 admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:24.521954 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90f1b8a-818a-4ef9-ae0e-4ac30497fc13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:24.522224 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90f1b8a-818a-4ef9-ae0e-4ac30497fc13 admin admin] PATH_INFO: `/v3/projects/f813e5dca6844d1cacdfa2d3214fb1f1/users/5d57dabc8b6144eb81afeab18217fd13/roles/3f2ca33902a04b6dbc2ef7d06d7289d3` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:24.537720 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90f1b8a-818a-4ef9-ae0e-4ac30497fc13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f813e5dca6844d1cacdfa2d3214fb1f1, user_id=5d57dabc8b6144eb81afeab18217fd13, role_id=3f2ca33902a04b6dbc2ef7d06d7289d3)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:24.540437 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90f1b8a-818a-4ef9-ae0e-4ac30497fc13 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:24.553444 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1410/2799] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:24:24 2022] PUT /identity/v3/projects/f813e5dca6844d1cacdfa2d3214fb1f1/users/5d57dabc8b6144eb81afeab18217fd13/roles/3f2ca33902a04b6dbc2ef7d06d7289d3 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:24:24.558271 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ade7c01-bf59-4080-9ad9-f07ad464cbe9 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:24.570449 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ade7c01-bf59-4080-9ad9-f07ad464cbe9 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:24.626990 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4444d3d8-e308-4f73-8c25-214487d6c563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:24.627608 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4444d3d8-e308-4f73-8c25-214487d6c563 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:24.627885 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4444d3d8-e308-4f73-8c25-214487d6c563 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:24.628155 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4444d3d8-e308-4f73-8c25-214487d6c563 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:24.628793 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4444d3d8-e308-4f73-8c25-214487d6c563 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:24.630508 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4444d3d8-e308-4f73-8c25-214487d6c563 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:24.641346 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1390/2800] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:24:24 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:24:24.645632 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90f1b8a-818a-4ef9-ae0e-4ac30497fc13 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:24.655849 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90f1b8a-818a-4ef9-ae0e-4ac30497fc13 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:24.660363 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68943080-0d00-48fa-9f14-13f7b65ea862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:24.660999 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68943080-0d00-48fa-9f14-13f7b65ea862 admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:24.661276 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68943080-0d00-48fa-9f14-13f7b65ea862 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:24.661544 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68943080-0d00-48fa-9f14-13f7b65ea862 admin admin] PATH_INFO: `/v3/projects/f813e5dca6844d1cacdfa2d3214fb1f1/users/5d57dabc8b6144eb81afeab18217fd13/roles/3f2ca33902a04b6dbc2ef7d06d7289d3` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:24.663471 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68943080-0d00-48fa-9f14-13f7b65ea862 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f813e5dca6844d1cacdfa2d3214fb1f1, user_id=5d57dabc8b6144eb81afeab18217fd13, role_id=3f2ca33902a04b6dbc2ef7d06d7289d3)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:24.666083 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68943080-0d00-48fa-9f14-13f7b65ea862 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:24.676472 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1411/2801] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:24:24 2022] PUT /identity/v3/projects/f813e5dca6844d1cacdfa2d3214fb1f1/users/5d57dabc8b6144eb81afeab18217fd13/roles/3f2ca33902a04b6dbc2ef7d06d7289d3 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:24:24.687171 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aea0e58-cf6c-4143-86e3-a2d550e5cf21 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:24.687457 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aea0e58-cf6c-4143-86e3-a2d550e5cf21 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:24.687725 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aea0e58-cf6c-4143-86e3-a2d550e5cf21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:24.715146 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-4aea0e58-cf6c-4143-86e3-a2d550e5cf21 None None] MFA Rules not processed for user `5d57dabc8b6144eb81afeab18217fd13`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:24:24.716295 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-4aea0e58-cf6c-4143-86e3-a2d550e5cf21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:24:24.781051 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:24:24.784967 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1391/2802] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:24:24 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:24:24.797356 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68943080-0d00-48fa-9f14-13f7b65ea862 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:24.799323 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-68943080-0d00-48fa-9f14-13f7b65ea862 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:24:24.877159 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68943080-0d00-48fa-9f14-13f7b65ea862 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:24.880784 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee96ddb2-c111-4e17-a03b-9da6c3ddf5bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535c258100df40b58a5ee44b2c21d621', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:24.881369 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee96ddb2-c111-4e17-a03b-9da6c3ddf5bb service cinder] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:24.881643 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee96ddb2-c111-4e17-a03b-9da6c3ddf5bb service cinder] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:24.881907 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee96ddb2-c111-4e17-a03b-9da6c3ddf5bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:24.890640 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee96ddb2-c111-4e17-a03b-9da6c3ddf5bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:24.892077 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee96ddb2-c111-4e17-a03b-9da6c3ddf5bb service cinder] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:24.901088 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1412/2803] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:24:24 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:24:24.966158 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aea0e58-cf6c-4143-86e3-a2d550e5cf21 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:25.026671 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aea0e58-cf6c-4143-86e3-a2d550e5cf21 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:25.031180 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d688e8a-3011-4e8d-8d6d-3fddc94e2a06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:25.031786 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d688e8a-3011-4e8d-8d6d-3fddc94e2a06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:25.032059 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d688e8a-3011-4e8d-8d6d-3fddc94e2a06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:25.032326 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d688e8a-3011-4e8d-8d6d-3fddc94e2a06 admin admin] PATH_INFO: `/v3/users/5ce9db5f04634742a34d99a3332831c3` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:25.033432 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d688e8a-3011-4e8d-8d6d-3fddc94e2a06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ce9db5f04634742a34d99a3332831c3)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:25.035365 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d688e8a-3011-4e8d-8d6d-3fddc94e2a06 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:25.086180 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-2d688e8a-3011-4e8d-8d6d-3fddc94e2a06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ce9db5f04634742a34d99a3332831c3'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:25.098771 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-2d688e8a-3011-4e8d-8d6d-3fddc94e2a06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ce9db5f04634742a34d99a3332831c3'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:25.106370 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-2d688e8a-3011-4e8d-8d6d-3fddc94e2a06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ce9db5f04634742a34d99a3332831c3'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:25.115285 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-2d688e8a-3011-4e8d-8d6d-3fddc94e2a06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ce9db5f04634742a34d99a3332831c3'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:25.115904 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-2d688e8a-3011-4e8d-8d6d-3fddc94e2a06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ce9db5f04634742a34d99a3332831c3'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:25.121925 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1392/2804] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 9 12:24:24 2022] DELETE /identity/v3/users/5ce9db5f04634742a34d99a3332831c3 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:24:25.126450 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90ee84b-3c27-4714-ac52-9b10c5720421 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:25.126744 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90ee84b-3c27-4714-ac52-9b10c5720421 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:25.127065 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90ee84b-3c27-4714-ac52-9b10c5720421 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:25.145469 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-d90ee84b-3c27-4714-ac52-9b10c5720421 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:24:25.146671 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-d90ee84b-3c27-4714-ac52-9b10c5720421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:24:25.204990 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1413/2805] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:24:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:24:25.214804 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d688e8a-3011-4e8d-8d6d-3fddc94e2a06 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:25.216849 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-2d688e8a-3011-4e8d-8d6d-3fddc94e2a06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:24:25.273651 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d688e8a-3011-4e8d-8d6d-3fddc94e2a06 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:25.277263 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08697e07-48a5-4622-a24d-a84c24553b36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:25.277860 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08697e07-48a5-4622-a24d-a84c24553b36 service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:25.278135 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08697e07-48a5-4622-a24d-a84c24553b36 service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:25.278951 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08697e07-48a5-4622-a24d-a84c24553b36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:25.287823 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08697e07-48a5-4622-a24d-a84c24553b36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:25.289239 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08697e07-48a5-4622-a24d-a84c24553b36 service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:25.299842 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1393/2806] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:24:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:24:25.570319 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90ee84b-3c27-4714-ac52-9b10c5720421 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:25.571809 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-d90ee84b-3c27-4714-ac52-9b10c5720421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:24:25.586276 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90ee84b-3c27-4714-ac52-9b10c5720421 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:25.591590 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a85bb7d-ed38-4847-b77b-54d1f3966a78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:25.592357 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a85bb7d-ed38-4847-b77b-54d1f3966a78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:25.592644 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a85bb7d-ed38-4847-b77b-54d1f3966a78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:25.592918 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a85bb7d-ed38-4847-b77b-54d1f3966a78 admin admin] PATH_INFO: `/v3/projects/25eab5f0def340079cec8987940c49c3` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:25.594186 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a85bb7d-ed38-4847-b77b-54d1f3966a78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=25eab5f0def340079cec8987940c49c3)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:25.596025 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a85bb7d-ed38-4847-b77b-54d1f3966a78 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:25.640094 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:24:25.643187 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-8a85bb7d-ed38-4847-b77b-54d1f3966a78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25eab5f0def340079cec8987940c49c3', 'request_id': 'req-8a85bb7d-ed38-4847-b77b-54d1f3966a78'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:25.650195 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-8a85bb7d-ed38-4847-b77b-54d1f3966a78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25eab5f0def340079cec8987940c49c3', 'request_id': 'req-8a85bb7d-ed38-4847-b77b-54d1f3966a78'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:25.657601 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-8a85bb7d-ed38-4847-b77b-54d1f3966a78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25eab5f0def340079cec8987940c49c3', 'request_id': 'req-8a85bb7d-ed38-4847-b77b-54d1f3966a78'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:25.669947 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-8a85bb7d-ed38-4847-b77b-54d1f3966a78 admin admin] The token cache is being invalidate because project 25eab5f0def340079cec8987940c49c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81591) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:24:25.670229 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-8a85bb7d-ed38-4847-b77b-54d1f3966a78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:25.670942 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-8a85bb7d-ed38-4847-b77b-54d1f3966a78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:25.672469 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1414/2807] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 9 12:24:25 2022] DELETE /identity/v3/projects/25eab5f0def340079cec8987940c49c3 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:24:25.676743 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08697e07-48a5-4622-a24d-a84c24553b36 service neutron] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:25.686603 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08697e07-48a5-4622-a24d-a84c24553b36 service neutron] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:25.704841 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:24:25.744255 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1febb3a1-ee28-4522-8c36-90c9623f07e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:25.744784 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1febb3a1-ee28-4522-8c36-90c9623f07e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:25.745065 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1febb3a1-ee28-4522-8c36-90c9623f07e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:25.745333 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1febb3a1-ee28-4522-8c36-90c9623f07e3 admin admin] PATH_INFO: `/v3/users/5d57dabc8b6144eb81afeab18217fd13` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:25.746446 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1febb3a1-ee28-4522-8c36-90c9623f07e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d57dabc8b6144eb81afeab18217fd13)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:25.748428 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1febb3a1-ee28-4522-8c36-90c9623f07e3 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:25.793571 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-1febb3a1-ee28-4522-8c36-90c9623f07e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d57dabc8b6144eb81afeab18217fd13'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:25.806133 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-1febb3a1-ee28-4522-8c36-90c9623f07e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d57dabc8b6144eb81afeab18217fd13'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:25.813365 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-1febb3a1-ee28-4522-8c36-90c9623f07e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d57dabc8b6144eb81afeab18217fd13'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:25.819905 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-1febb3a1-ee28-4522-8c36-90c9623f07e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d57dabc8b6144eb81afeab18217fd13'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:25.820628 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-1febb3a1-ee28-4522-8c36-90c9623f07e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d57dabc8b6144eb81afeab18217fd13'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:25.827477 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1394/2808] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 9 12:24:25 2022] DELETE /identity/v3/users/5d57dabc8b6144eb81afeab18217fd13 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:24:26.100145 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a85bb7d-ed38-4847-b77b-54d1f3966a78 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:26.101576 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-8a85bb7d-ed38-4847-b77b-54d1f3966a78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:24:26.116020 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a85bb7d-ed38-4847-b77b-54d1f3966a78 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:26.173811 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b30733-ed99-48ee-9284-760ef376cda9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:26.174431 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b30733-ed99-48ee-9284-760ef376cda9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:26.174705 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b30733-ed99-48ee-9284-760ef376cda9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:26.175004 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b30733-ed99-48ee-9284-760ef376cda9 admin admin] PATH_INFO: `/v3/projects/f813e5dca6844d1cacdfa2d3214fb1f1` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:26.176112 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b30733-ed99-48ee-9284-760ef376cda9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f813e5dca6844d1cacdfa2d3214fb1f1)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:26.178009 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b30733-ed99-48ee-9284-760ef376cda9 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:26.223779 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-04b30733-ed99-48ee-9284-760ef376cda9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f813e5dca6844d1cacdfa2d3214fb1f1', 'request_id': 'req-04b30733-ed99-48ee-9284-760ef376cda9'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:26.229431 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-04b30733-ed99-48ee-9284-760ef376cda9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f813e5dca6844d1cacdfa2d3214fb1f1', 'request_id': 'req-04b30733-ed99-48ee-9284-760ef376cda9'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:26.236152 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-04b30733-ed99-48ee-9284-760ef376cda9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f813e5dca6844d1cacdfa2d3214fb1f1', 'request_id': 'req-04b30733-ed99-48ee-9284-760ef376cda9'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:26.248797 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-04b30733-ed99-48ee-9284-760ef376cda9 admin admin] The token cache is being invalidate because project f813e5dca6844d1cacdfa2d3214fb1f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81591) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:24:26.249080 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-04b30733-ed99-48ee-9284-760ef376cda9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:26.249702 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-04b30733-ed99-48ee-9284-760ef376cda9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:26.251241 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1415/2809] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 9 12:24:26 2022] DELETE /identity/v3/projects/f813e5dca6844d1cacdfa2d3214fb1f1 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:24:26.256150 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd8f08c-be4f-4f46-ad95-6651ce653ba4 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:26.256601 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd8f08c-be4f-4f46-ad95-6651ce653ba4 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:26.256718 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd8f08c-be4f-4f46-ad95-6651ce653ba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:26.276222 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-8cd8f08c-be4f-4f46-ad95-6651ce653ba4 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:24:26.277350 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-8cd8f08c-be4f-4f46-ad95-6651ce653ba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:24:26.336258 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1395/2810] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:24:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:24:26.344135 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfaf14c-d1e4-4935-9f52-289a016e1b97 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:26.344443 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfaf14c-d1e4-4935-9f52-289a016e1b97 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:26.344711 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfaf14c-d1e4-4935-9f52-289a016e1b97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:26.364544 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-1cfaf14c-d1e4-4935-9f52-289a016e1b97 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:24:26.365742 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-1cfaf14c-d1e4-4935-9f52-289a016e1b97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:24:26.374273 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1416/2811] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:24:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:24:26.378466 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd8f08c-be4f-4f46-ad95-6651ce653ba4 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:26.388609 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd8f08c-be4f-4f46-ad95-6651ce653ba4 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:26.393129 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb52620-652b-400f-9618-7027e9e3736b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:26.393717 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb52620-652b-400f-9618-7027e9e3736b admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:26.393991 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb52620-652b-400f-9618-7027e9e3736b admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:26.394257 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb52620-652b-400f-9618-7027e9e3736b admin admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:26.394924 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb52620-652b-400f-9618-7027e9e3736b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:24:26.395238 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb52620-652b-400f-9618-7027e9e3736b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:26.396984 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb52620-652b-400f-9618-7027e9e3736b admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:26.408435 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1396/2812] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 9 12:24:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:24:26.412643 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfaf14c-d1e4-4935-9f52-289a016e1b97 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:26.422635 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfaf14c-d1e4-4935-9f52-289a016e1b97 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:26.427450 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be58d967-3ca6-42c3-bac2-83f9b9aa6d67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:26.428030 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be58d967-3ca6-42c3-bac2-83f9b9aa6d67 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:26.428303 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be58d967-3ca6-42c3-bac2-83f9b9aa6d67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:26.428569 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be58d967-3ca6-42c3-bac2-83f9b9aa6d67 admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:26.429192 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be58d967-3ca6-42c3-bac2-83f9b9aa6d67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:26.431088 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be58d967-3ca6-42c3-bac2-83f9b9aa6d67 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:26.460178 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1417/2813] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 9 12:24:26 2022] POST /identity/v3/projects => generated 379 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:24:26.464263 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb52620-652b-400f-9618-7027e9e3736b admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:26.474562 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb52620-652b-400f-9618-7027e9e3736b admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:26.500572 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:24:26.531019 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a373b8f-8503-42bc-a135-1ccc62384fb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:26.531558 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a373b8f-8503-42bc-a135-1ccc62384fb4 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:26.531835 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a373b8f-8503-42bc-a135-1ccc62384fb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:26.532107 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a373b8f-8503-42bc-a135-1ccc62384fb4 admin admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:26.532738 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a373b8f-8503-42bc-a135-1ccc62384fb4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:26.534639 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a373b8f-8503-42bc-a135-1ccc62384fb4 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:26.556538 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1397/2814] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:24:26 2022] POST /identity/v3/users => generated 290 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:24:26.561534 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be58d967-3ca6-42c3-bac2-83f9b9aa6d67 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:26.571502 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be58d967-3ca6-42c3-bac2-83f9b9aa6d67 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:26.576334 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889d8b97-1cca-43b9-8643-3e8b75981ebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:26.576922 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889d8b97-1cca-43b9-8643-3e8b75981ebc admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:26.577224 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889d8b97-1cca-43b9-8643-3e8b75981ebc admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:26.577491 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889d8b97-1cca-43b9-8643-3e8b75981ebc admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:26.578117 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889d8b97-1cca-43b9-8643-3e8b75981ebc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:26.579729 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889d8b97-1cca-43b9-8643-3e8b75981ebc admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:26.589348 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1418/2815] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:24:26 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:24:26.593687 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a373b8f-8503-42bc-a135-1ccc62384fb4 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:26.603545 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a373b8f-8503-42bc-a135-1ccc62384fb4 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:26.608833 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01024513-e6e7-48aa-955c-8998e5de9292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:26.609412 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01024513-e6e7-48aa-955c-8998e5de9292 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:26.609690 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01024513-e6e7-48aa-955c-8998e5de9292 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:26.609958 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01024513-e6e7-48aa-955c-8998e5de9292 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:26.610586 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01024513-e6e7-48aa-955c-8998e5de9292 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:26.612481 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01024513-e6e7-48aa-955c-8998e5de9292 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:26.624017 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1398/2816] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:24:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:24:26.628439 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889d8b97-1cca-43b9-8643-3e8b75981ebc admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:26.638740 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889d8b97-1cca-43b9-8643-3e8b75981ebc admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:26.643305 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d17a9c9-10f5-48ac-ab3c-ffaed7baed03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:26.643953 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d17a9c9-10f5-48ac-ab3c-ffaed7baed03 admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:26.644227 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d17a9c9-10f5-48ac-ab3c-ffaed7baed03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:26.644498 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d17a9c9-10f5-48ac-ab3c-ffaed7baed03 admin admin] PATH_INFO: `/v3/projects/a79913ccb1ad41f38cb48d0cd8cdbd28/users/3243f99cc64842a682553645317334bc/roles/eecd2f160dd844f5906f3d345780c501` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:26.659826 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d17a9c9-10f5-48ac-ab3c-ffaed7baed03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a79913ccb1ad41f38cb48d0cd8cdbd28, user_id=3243f99cc64842a682553645317334bc, role_id=eecd2f160dd844f5906f3d345780c501)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:26.662457 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d17a9c9-10f5-48ac-ab3c-ffaed7baed03 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:26.671829 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:24:26.678329 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1419/2817] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:24:26 2022] PUT /identity/v3/projects/a79913ccb1ad41f38cb48d0cd8cdbd28/users/3243f99cc64842a682553645317334bc/roles/eecd2f160dd844f5906f3d345780c501 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:24:26.690437 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c423fb-bf71-423f-a070-66c4bda3a87d None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:26.690826 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c423fb-bf71-423f-a070-66c4bda3a87d None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:26.691182 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c423fb-bf71-423f-a070-66c4bda3a87d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:26.720484 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-e7c423fb-bf71-423f-a070-66c4bda3a87d None None] MFA Rules not processed for user `3243f99cc64842a682553645317334bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:24:26.721675 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-e7c423fb-bf71-423f-a070-66c4bda3a87d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:24:26.785191 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1399/2818] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:24:26 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 12:24:26.789626 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d17a9c9-10f5-48ac-ab3c-ffaed7baed03 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:26.850510 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d17a9c9-10f5-48ac-ab3c-ffaed7baed03 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:26.855218 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9152cb-da16-4450-aee9-9278c2555bfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:26.855814 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9152cb-da16-4450-aee9-9278c2555bfc admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:26.856095 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9152cb-da16-4450-aee9-9278c2555bfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:26.856368 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9152cb-da16-4450-aee9-9278c2555bfc admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:26.857762 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9152cb-da16-4450-aee9-9278c2555bfc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:26.859698 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9152cb-da16-4450-aee9-9278c2555bfc admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:26.885343 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1420/2819] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 9 12:24:26 2022] POST /identity/v3/projects => generated 381 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:24:26.889442 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c423fb-bf71-423f-a070-66c4bda3a87d None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:26.899862 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c423fb-bf71-423f-a070-66c4bda3a87d None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:26.956212 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766d2ef3-0e3f-4c6e-9100-4b379d343fc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:26.956814 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766d2ef3-0e3f-4c6e-9100-4b379d343fc9 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:26.957088 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766d2ef3-0e3f-4c6e-9100-4b379d343fc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:26.957386 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766d2ef3-0e3f-4c6e-9100-4b379d343fc9 admin admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:26.958016 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766d2ef3-0e3f-4c6e-9100-4b379d343fc9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:26.960039 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766d2ef3-0e3f-4c6e-9100-4b379d343fc9 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:26.981426 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1400/2820] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:24:26 2022] POST /identity/v3/users => generated 297 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:24:26.986088 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9152cb-da16-4450-aee9-9278c2555bfc admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:26.996143 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9152cb-da16-4450-aee9-9278c2555bfc admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:27.001194 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5a9362-4ebf-498e-89ba-52414ea4b56e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:27.001782 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5a9362-4ebf-498e-89ba-52414ea4b56e admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:27.002059 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5a9362-4ebf-498e-89ba-52414ea4b56e admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:27.002328 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5a9362-4ebf-498e-89ba-52414ea4b56e admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:27.003016 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5a9362-4ebf-498e-89ba-52414ea4b56e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:27.004585 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5a9362-4ebf-498e-89ba-52414ea4b56e admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:27.016535 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1421/2821] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:24:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:24:27.020653 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766d2ef3-0e3f-4c6e-9100-4b379d343fc9 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:27.031344 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766d2ef3-0e3f-4c6e-9100-4b379d343fc9 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:27.035999 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87127890-64c9-4e24-a743-8659755f2910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:27.036588 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87127890-64c9-4e24-a743-8659755f2910 admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:27.036869 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87127890-64c9-4e24-a743-8659755f2910 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:27.037143 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87127890-64c9-4e24-a743-8659755f2910 admin admin] PATH_INFO: `/v3/projects/fef8244bb6bb47ac8273cd0bdf5dcd50/users/2dd5ab0fe47c4eeda302cf37e568fa7c/roles/3f2ca33902a04b6dbc2ef7d06d7289d3` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:27.051799 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87127890-64c9-4e24-a743-8659755f2910 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fef8244bb6bb47ac8273cd0bdf5dcd50, user_id=2dd5ab0fe47c4eeda302cf37e568fa7c, role_id=3f2ca33902a04b6dbc2ef7d06d7289d3)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:27.054490 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87127890-64c9-4e24-a743-8659755f2910 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:27.067080 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1401/2822] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:24:27 2022] PUT /identity/v3/projects/fef8244bb6bb47ac8273cd0bdf5dcd50/users/2dd5ab0fe47c4eeda302cf37e568fa7c/roles/3f2ca33902a04b6dbc2ef7d06d7289d3 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:24:27.070972 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5a9362-4ebf-498e-89ba-52414ea4b56e admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:27.081337 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5a9362-4ebf-498e-89ba-52414ea4b56e admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:27.136011 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c806b89b-f3ca-4a3e-82a6-50144dec6697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:27.136610 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c806b89b-f3ca-4a3e-82a6-50144dec6697 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:27.136886 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c806b89b-f3ca-4a3e-82a6-50144dec6697 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:27.137153 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c806b89b-f3ca-4a3e-82a6-50144dec6697 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:27.138348 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c806b89b-f3ca-4a3e-82a6-50144dec6697 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:27.139944 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c806b89b-f3ca-4a3e-82a6-50144dec6697 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:27.149327 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1422/2823] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:24:27 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:24:27.154168 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87127890-64c9-4e24-a743-8659755f2910 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:27.159525 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:24:27.165686 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87127890-64c9-4e24-a743-8659755f2910 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:27.171083 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f14c211-6149-415b-ab4e-c430953cd925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:27.171734 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f14c211-6149-415b-ab4e-c430953cd925 admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:27.172011 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f14c211-6149-415b-ab4e-c430953cd925 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:27.172283 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f14c211-6149-415b-ab4e-c430953cd925 admin admin] PATH_INFO: `/v3/projects/fef8244bb6bb47ac8273cd0bdf5dcd50/users/2dd5ab0fe47c4eeda302cf37e568fa7c/roles/3f2ca33902a04b6dbc2ef7d06d7289d3` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:27.174481 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f14c211-6149-415b-ab4e-c430953cd925 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fef8244bb6bb47ac8273cd0bdf5dcd50, user_id=2dd5ab0fe47c4eeda302cf37e568fa7c, role_id=3f2ca33902a04b6dbc2ef7d06d7289d3)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:27.177215 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f14c211-6149-415b-ab4e-c430953cd925 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:27.187375 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1402/2824] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:24:27 2022] PUT /identity/v3/projects/fef8244bb6bb47ac8273cd0bdf5dcd50/users/2dd5ab0fe47c4eeda302cf37e568fa7c/roles/3f2ca33902a04b6dbc2ef7d06d7289d3 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:24:27.197352 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cd01ea-ffa9-4b0b-84ab-9161f37a7375 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:27.197636 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cd01ea-ffa9-4b0b-84ab-9161f37a7375 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:27.197909 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cd01ea-ffa9-4b0b-84ab-9161f37a7375 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:27.225912 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-35cd01ea-ffa9-4b0b-84ab-9161f37a7375 None None] MFA Rules not processed for user `2dd5ab0fe47c4eeda302cf37e568fa7c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:24:27.227145 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-35cd01ea-ffa9-4b0b-84ab-9161f37a7375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:24:27.284215 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:24:27.293483 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1423/2825] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:24:27 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:24:27.305790 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f14c211-6149-415b-ab4e-c430953cd925 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:27.307591 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-6f14c211-6149-415b-ab4e-c430953cd925 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:24:27.364093 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f14c211-6149-415b-ab4e-c430953cd925 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:27.367738 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ff8d4c-801d-4949-b3e7-e79589161981 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535c258100df40b58a5ee44b2c21d621', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:27.368333 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ff8d4c-801d-4949-b3e7-e79589161981 service cinder] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:27.368608 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ff8d4c-801d-4949-b3e7-e79589161981 service cinder] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:27.368876 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ff8d4c-801d-4949-b3e7-e79589161981 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:27.378203 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ff8d4c-801d-4949-b3e7-e79589161981 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:27.379650 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ff8d4c-801d-4949-b3e7-e79589161981 service cinder] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:27.389403 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1403/2826] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:24:27 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:24:27.695798 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cd01ea-ffa9-4b0b-84ab-9161f37a7375 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:27.705420 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cd01ea-ffa9-4b0b-84ab-9161f37a7375 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:27.709912 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed416ea6-f957-4408-b839-82cb33e919f6 tempest-VolumeQuotaClassesTest-1672766775 tempest-VolumeQuotaClassesTest-1672766775-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd5ab0fe47c4eeda302cf37e568fa7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fef8244bb6bb47ac8273cd0bdf5dcd50', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:27.710514 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed416ea6-f957-4408-b839-82cb33e919f6 tempest-VolumeQuotaClassesTest-1672766775 tempest-VolumeQuotaClassesTest-1672766775-project-admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:27.710792 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed416ea6-f957-4408-b839-82cb33e919f6 tempest-VolumeQuotaClassesTest-1672766775 tempest-VolumeQuotaClassesTest-1672766775-project-admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:27.711095 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed416ea6-f957-4408-b839-82cb33e919f6 tempest-VolumeQuotaClassesTest-1672766775 tempest-VolumeQuotaClassesTest-1672766775-project-admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:27.711737 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed416ea6-f957-4408-b839-82cb33e919f6 tempest-VolumeQuotaClassesTest-1672766775 tempest-VolumeQuotaClassesTest-1672766775-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:24:27.712038 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed416ea6-f957-4408-b839-82cb33e919f6 tempest-VolumeQuotaClassesTest-1672766775 tempest-VolumeQuotaClassesTest-1672766775-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:27.713647 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed416ea6-f957-4408-b839-82cb33e919f6 tempest-VolumeQuotaClassesTest-1672766775 tempest-VolumeQuotaClassesTest-1672766775-project-admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:27.726122 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1424/2827] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 9 12:24:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:24:27.730360 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ff8d4c-801d-4949-b3e7-e79589161981 service cinder] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:27.740065 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ff8d4c-801d-4949-b3e7-e79589161981 service cinder] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:27.744652 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43535f09-0bab-49ab-b929-86a8b867dfcc tempest-VolumeQuotaClassesTest-1672766775 tempest-VolumeQuotaClassesTest-1672766775-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd5ab0fe47c4eeda302cf37e568fa7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fef8244bb6bb47ac8273cd0bdf5dcd50', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:27.745220 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43535f09-0bab-49ab-b929-86a8b867dfcc tempest-VolumeQuotaClassesTest-1672766775 tempest-VolumeQuotaClassesTest-1672766775-project-admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:27.745493 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43535f09-0bab-49ab-b929-86a8b867dfcc tempest-VolumeQuotaClassesTest-1672766775 tempest-VolumeQuotaClassesTest-1672766775-project-admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:27.745757 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43535f09-0bab-49ab-b929-86a8b867dfcc tempest-VolumeQuotaClassesTest-1672766775 tempest-VolumeQuotaClassesTest-1672766775-project-admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:27.746393 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43535f09-0bab-49ab-b929-86a8b867dfcc tempest-VolumeQuotaClassesTest-1672766775 tempest-VolumeQuotaClassesTest-1672766775-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:27.748928 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43535f09-0bab-49ab-b929-86a8b867dfcc tempest-VolumeQuotaClassesTest-1672766775 tempest-VolumeQuotaClassesTest-1672766775-project-admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:27.774612 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1404/2828] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 9 12:24:27 2022] POST /identity/v3/projects => generated 355 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:24:27.778687 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed416ea6-f957-4408-b839-82cb33e919f6 tempest-VolumeQuotaClassesTest-1672766775 tempest-VolumeQuotaClassesTest-1672766775-project-admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:27.788433 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed416ea6-f957-4408-b839-82cb33e919f6 tempest-VolumeQuotaClassesTest-1672766775 tempest-VolumeQuotaClassesTest-1672766775-project-admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:27.843971 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb85939-0bb2-4d1a-a97d-0b5570250d72 tempest-VolumeQuotaClassesTest-1672766775 tempest-VolumeQuotaClassesTest-1672766775-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd5ab0fe47c4eeda302cf37e568fa7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fef8244bb6bb47ac8273cd0bdf5dcd50', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:27.844574 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb85939-0bb2-4d1a-a97d-0b5570250d72 tempest-VolumeQuotaClassesTest-1672766775 tempest-VolumeQuotaClassesTest-1672766775-project-admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:27.844857 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb85939-0bb2-4d1a-a97d-0b5570250d72 tempest-VolumeQuotaClassesTest-1672766775 tempest-VolumeQuotaClassesTest-1672766775-project-admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:27.845134 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb85939-0bb2-4d1a-a97d-0b5570250d72 tempest-VolumeQuotaClassesTest-1672766775 tempest-VolumeQuotaClassesTest-1672766775-project-admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:27.845766 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb85939-0bb2-4d1a-a97d-0b5570250d72 tempest-VolumeQuotaClassesTest-1672766775 tempest-VolumeQuotaClassesTest-1672766775-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:24:27.846070 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb85939-0bb2-4d1a-a97d-0b5570250d72 tempest-VolumeQuotaClassesTest-1672766775 tempest-VolumeQuotaClassesTest-1672766775-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:27.847688 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb85939-0bb2-4d1a-a97d-0b5570250d72 tempest-VolumeQuotaClassesTest-1672766775 tempest-VolumeQuotaClassesTest-1672766775-project-admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:27.859781 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1425/2829] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 9 12:24:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:24:27.896767 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43535f09-0bab-49ab-b929-86a8b867dfcc tempest-VolumeQuotaClassesTest-1672766775 tempest-VolumeQuotaClassesTest-1672766775-project-admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:27.907195 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43535f09-0bab-49ab-b929-86a8b867dfcc tempest-VolumeQuotaClassesTest-1672766775 tempest-VolumeQuotaClassesTest-1672766775-project-admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:27.911741 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c62c3aa-e45e-4a4a-9aec-7d69dc56b332 tempest-VolumeQuotaClassesTest-1672766775 tempest-VolumeQuotaClassesTest-1672766775-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dd5ab0fe47c4eeda302cf37e568fa7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fef8244bb6bb47ac8273cd0bdf5dcd50', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:27.912350 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c62c3aa-e45e-4a4a-9aec-7d69dc56b332 tempest-VolumeQuotaClassesTest-1672766775 tempest-VolumeQuotaClassesTest-1672766775-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:27.912633 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c62c3aa-e45e-4a4a-9aec-7d69dc56b332 tempest-VolumeQuotaClassesTest-1672766775 tempest-VolumeQuotaClassesTest-1672766775-project-admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:27.912902 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c62c3aa-e45e-4a4a-9aec-7d69dc56b332 tempest-VolumeQuotaClassesTest-1672766775 tempest-VolumeQuotaClassesTest-1672766775-project-admin] PATH_INFO: `/v3/projects/00474df772f74d0aac60eeb847c34e3b` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:27.914028 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c62c3aa-e45e-4a4a-9aec-7d69dc56b332 tempest-VolumeQuotaClassesTest-1672766775 tempest-VolumeQuotaClassesTest-1672766775-project-admin] RBAC: Authorizing `identity:delete_project(project_id=00474df772f74d0aac60eeb847c34e3b)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:27.915974 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c62c3aa-e45e-4a4a-9aec-7d69dc56b332 tempest-VolumeQuotaClassesTest-1672766775 tempest-VolumeQuotaClassesTest-1672766775-project-admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:27.962945 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-5c62c3aa-e45e-4a4a-9aec-7d69dc56b332 tempest-VolumeQuotaClassesTest-1672766775 tempest-VolumeQuotaClassesTest-1672766775-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00474df772f74d0aac60eeb847c34e3b', 'request_id': 'req-5c62c3aa-e45e-4a4a-9aec-7d69dc56b332'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:27.968590 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-5c62c3aa-e45e-4a4a-9aec-7d69dc56b332 tempest-VolumeQuotaClassesTest-1672766775 tempest-VolumeQuotaClassesTest-1672766775-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00474df772f74d0aac60eeb847c34e3b', 'request_id': 'req-5c62c3aa-e45e-4a4a-9aec-7d69dc56b332'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:27.975746 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-5c62c3aa-e45e-4a4a-9aec-7d69dc56b332 tempest-VolumeQuotaClassesTest-1672766775 tempest-VolumeQuotaClassesTest-1672766775-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00474df772f74d0aac60eeb847c34e3b', 'request_id': 'req-5c62c3aa-e45e-4a4a-9aec-7d69dc56b332'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:27.987816 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-5c62c3aa-e45e-4a4a-9aec-7d69dc56b332 tempest-VolumeQuotaClassesTest-1672766775 tempest-VolumeQuotaClassesTest-1672766775-project-admin] The token cache is being invalidate because project 00474df772f74d0aac60eeb847c34e3b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:24:27.988099 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-5c62c3aa-e45e-4a4a-9aec-7d69dc56b332 tempest-VolumeQuotaClassesTest-1672766775 tempest-VolumeQuotaClassesTest-1672766775-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:27.988777 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-5c62c3aa-e45e-4a4a-9aec-7d69dc56b332 tempest-VolumeQuotaClassesTest-1672766775 tempest-VolumeQuotaClassesTest-1672766775-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:27.991055 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1405/2830] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 9 12:24:27 2022] DELETE /identity/v3/projects/00474df772f74d0aac60eeb847c34e3b => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:24:28.182852 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb85939-0bb2-4d1a-a97d-0b5570250d72 tempest-VolumeQuotaClassesTest-1672766775 tempest-VolumeQuotaClassesTest-1672766775-project-admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:28.184606 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-bcb85939-0bb2-4d1a-a97d-0b5570250d72 tempest-VolumeQuotaClassesTest-1672766775 tempest-VolumeQuotaClassesTest-1672766775-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:24:28.249263 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb85939-0bb2-4d1a-a97d-0b5570250d72 tempest-VolumeQuotaClassesTest-1672766775 tempest-VolumeQuotaClassesTest-1672766775-project-admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:28.304172 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cadbea-f694-42c2-8d2a-8220ef47c180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:28.304799 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cadbea-f694-42c2-8d2a-8220ef47c180 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:28.305076 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cadbea-f694-42c2-8d2a-8220ef47c180 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:28.305345 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cadbea-f694-42c2-8d2a-8220ef47c180 admin admin] PATH_INFO: `/v3/users/3243f99cc64842a682553645317334bc` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:28.306466 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cadbea-f694-42c2-8d2a-8220ef47c180 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3243f99cc64842a682553645317334bc)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:28.308405 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cadbea-f694-42c2-8d2a-8220ef47c180 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:28.334361 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:24:28.353829 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-63cadbea-f694-42c2-8d2a-8220ef47c180 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3243f99cc64842a682553645317334bc'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:28.361500 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-63cadbea-f694-42c2-8d2a-8220ef47c180 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3243f99cc64842a682553645317334bc'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:28.373427 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-63cadbea-f694-42c2-8d2a-8220ef47c180 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3243f99cc64842a682553645317334bc'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:28.380549 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-63cadbea-f694-42c2-8d2a-8220ef47c180 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3243f99cc64842a682553645317334bc'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:28.381183 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-63cadbea-f694-42c2-8d2a-8220ef47c180 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3243f99cc64842a682553645317334bc'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:28.387002 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1426/2831] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 9 12:24:28 2022] DELETE /identity/v3/users/3243f99cc64842a682553645317334bc => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:24:28.391561 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20295e7-07df-41ea-92c5-296059024dec None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:28.392027 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20295e7-07df-41ea-92c5-296059024dec None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:28.392149 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20295e7-07df-41ea-92c5-296059024dec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:28.410786 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-f20295e7-07df-41ea-92c5-296059024dec None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:24:28.412001 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-f20295e7-07df-41ea-92c5-296059024dec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:24:28.457606 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:24:28.471796 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1406/2832] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:24:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:24:28.481489 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cadbea-f694-42c2-8d2a-8220ef47c180 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:28.483723 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-63cadbea-f694-42c2-8d2a-8220ef47c180 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:24:28.539753 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cadbea-f694-42c2-8d2a-8220ef47c180 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:28.543531 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae813e7-b746-40d0-9170-9b40c59c1219 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:28.544265 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae813e7-b746-40d0-9170-9b40c59c1219 service neutron] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:28.544555 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae813e7-b746-40d0-9170-9b40c59c1219 service neutron] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:28.544831 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae813e7-b746-40d0-9170-9b40c59c1219 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:28.554139 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae813e7-b746-40d0-9170-9b40c59c1219 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:28.555600 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae813e7-b746-40d0-9170-9b40c59c1219 service neutron] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:28.565653 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1427/2833] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:24:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:24:28.831839 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20295e7-07df-41ea-92c5-296059024dec None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:28.833288 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-f20295e7-07df-41ea-92c5-296059024dec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:24:28.847284 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20295e7-07df-41ea-92c5-296059024dec None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:28.851835 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f7222c-d216-4cd4-8421-18d9d36f9c1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:28.852530 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f7222c-d216-4cd4-8421-18d9d36f9c1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:28.852729 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f7222c-d216-4cd4-8421-18d9d36f9c1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:28.852996 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f7222c-d216-4cd4-8421-18d9d36f9c1d admin admin] PATH_INFO: `/v3/projects/a79913ccb1ad41f38cb48d0cd8cdbd28` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:28.854103 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f7222c-d216-4cd4-8421-18d9d36f9c1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=a79913ccb1ad41f38cb48d0cd8cdbd28)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:28.856083 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f7222c-d216-4cd4-8421-18d9d36f9c1d admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:28.907168 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-a3f7222c-d216-4cd4-8421-18d9d36f9c1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a79913ccb1ad41f38cb48d0cd8cdbd28', 'request_id': 'req-a3f7222c-d216-4cd4-8421-18d9d36f9c1d'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:28.912688 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-a3f7222c-d216-4cd4-8421-18d9d36f9c1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a79913ccb1ad41f38cb48d0cd8cdbd28', 'request_id': 'req-a3f7222c-d216-4cd4-8421-18d9d36f9c1d'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:28.920856 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-a3f7222c-d216-4cd4-8421-18d9d36f9c1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a79913ccb1ad41f38cb48d0cd8cdbd28', 'request_id': 'req-a3f7222c-d216-4cd4-8421-18d9d36f9c1d'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:28.932268 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-a3f7222c-d216-4cd4-8421-18d9d36f9c1d admin admin] The token cache is being invalidate because project a79913ccb1ad41f38cb48d0cd8cdbd28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:24:28.932554 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-a3f7222c-d216-4cd4-8421-18d9d36f9c1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:28.933173 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-a3f7222c-d216-4cd4-8421-18d9d36f9c1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:28.935173 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1407/2834] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 9 12:24:28 2022] DELETE /identity/v3/projects/a79913ccb1ad41f38cb48d0cd8cdbd28 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:24:28.939504 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae813e7-b746-40d0-9170-9b40c59c1219 service neutron] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:28.949014 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae813e7-b746-40d0-9170-9b40c59c1219 service neutron] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:29.004213 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d0d9ba-f294-466a-9de2-76cd0946089e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:29.004850 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d0d9ba-f294-466a-9de2-76cd0946089e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:29.005131 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d0d9ba-f294-466a-9de2-76cd0946089e admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:29.005404 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d0d9ba-f294-466a-9de2-76cd0946089e admin admin] PATH_INFO: `/v3/users/2dd5ab0fe47c4eeda302cf37e568fa7c` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:29.006517 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d0d9ba-f294-466a-9de2-76cd0946089e admin admin] RBAC: Authorizing `identity:delete_user(user_id=2dd5ab0fe47c4eeda302cf37e568fa7c)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:29.008484 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d0d9ba-f294-466a-9de2-76cd0946089e admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:29.054115 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-96d0d9ba-f294-466a-9de2-76cd0946089e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2dd5ab0fe47c4eeda302cf37e568fa7c'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:29.055291 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:24:29.064190 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-96d0d9ba-f294-466a-9de2-76cd0946089e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2dd5ab0fe47c4eeda302cf37e568fa7c'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:29.076610 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-96d0d9ba-f294-466a-9de2-76cd0946089e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2dd5ab0fe47c4eeda302cf37e568fa7c'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:29.083704 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-96d0d9ba-f294-466a-9de2-76cd0946089e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2dd5ab0fe47c4eeda302cf37e568fa7c'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:29.084314 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-96d0d9ba-f294-466a-9de2-76cd0946089e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2dd5ab0fe47c4eeda302cf37e568fa7c'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:29.090024 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1428/2835] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 9 12:24:28 2022] DELETE /identity/v3/users/2dd5ab0fe47c4eeda302cf37e568fa7c => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:24:29.362610 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f7222c-d216-4cd4-8421-18d9d36f9c1d admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:29.364325 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-a3f7222c-d216-4cd4-8421-18d9d36f9c1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:24:29.378540 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f7222c-d216-4cd4-8421-18d9d36f9c1d admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:29.437232 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486e7644-79e1-4e66-82af-41f7b35cba48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:29.437906 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486e7644-79e1-4e66-82af-41f7b35cba48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:29.438111 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486e7644-79e1-4e66-82af-41f7b35cba48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:29.438395 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486e7644-79e1-4e66-82af-41f7b35cba48 admin admin] PATH_INFO: `/v3/projects/fef8244bb6bb47ac8273cd0bdf5dcd50` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:29.439498 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486e7644-79e1-4e66-82af-41f7b35cba48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fef8244bb6bb47ac8273cd0bdf5dcd50)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:29.441390 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486e7644-79e1-4e66-82af-41f7b35cba48 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:29.487237 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-486e7644-79e1-4e66-82af-41f7b35cba48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fef8244bb6bb47ac8273cd0bdf5dcd50', 'request_id': 'req-486e7644-79e1-4e66-82af-41f7b35cba48'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:29.492676 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-486e7644-79e1-4e66-82af-41f7b35cba48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fef8244bb6bb47ac8273cd0bdf5dcd50', 'request_id': 'req-486e7644-79e1-4e66-82af-41f7b35cba48'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:29.501279 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-486e7644-79e1-4e66-82af-41f7b35cba48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fef8244bb6bb47ac8273cd0bdf5dcd50', 'request_id': 'req-486e7644-79e1-4e66-82af-41f7b35cba48'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:29.512117 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-486e7644-79e1-4e66-82af-41f7b35cba48 admin admin] The token cache is being invalidate because project fef8244bb6bb47ac8273cd0bdf5dcd50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:24:29.512399 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-486e7644-79e1-4e66-82af-41f7b35cba48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:29.513074 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-486e7644-79e1-4e66-82af-41f7b35cba48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:29.514762 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1408/2836] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 9 12:24:29 2022] DELETE /identity/v3/projects/fef8244bb6bb47ac8273cd0bdf5dcd50 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:24:29.519642 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebea5cd2-87a1-4187-a646-2a7d94428ec0 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:29.519927 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebea5cd2-87a1-4187-a646-2a7d94428ec0 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:29.520245 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebea5cd2-87a1-4187-a646-2a7d94428ec0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:29.540810 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-ebea5cd2-87a1-4187-a646-2a7d94428ec0 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:24:29.541937 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-ebea5cd2-87a1-4187-a646-2a7d94428ec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:24:29.604105 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1429/2837] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:24:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:24:29.609848 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1704a99-3e3a-4f75-a25f-71a9d69601e9 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:29.610126 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1704a99-3e3a-4f75-a25f-71a9d69601e9 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:29.610392 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1704a99-3e3a-4f75-a25f-71a9d69601e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:29.624523 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:24:29.630762 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-d1704a99-3e3a-4f75-a25f-71a9d69601e9 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:24:29.632425 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-d1704a99-3e3a-4f75-a25f-71a9d69601e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:24:29.641517 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1409/2838] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:24:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:24:29.645526 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebea5cd2-87a1-4187-a646-2a7d94428ec0 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:29.655733 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebea5cd2-87a1-4187-a646-2a7d94428ec0 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:29.660206 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46a6d1f-0091-4105-9ddc-7a1c98ccf132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:29.660796 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46a6d1f-0091-4105-9ddc-7a1c98ccf132 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:29.661072 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46a6d1f-0091-4105-9ddc-7a1c98ccf132 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:29.661339 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46a6d1f-0091-4105-9ddc-7a1c98ccf132 admin admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:29.661974 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46a6d1f-0091-4105-9ddc-7a1c98ccf132 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:24:29.662273 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46a6d1f-0091-4105-9ddc-7a1c98ccf132 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:29.663878 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46a6d1f-0091-4105-9ddc-7a1c98ccf132 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:29.676016 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1430/2839] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 9 12:24:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:24:29.680346 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1704a99-3e3a-4f75-a25f-71a9d69601e9 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:29.689755 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1704a99-3e3a-4f75-a25f-71a9d69601e9 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:29.694279 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e799d3-10c9-4938-aa42-8258ddb7946b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:29.694860 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e799d3-10c9-4938-aa42-8258ddb7946b admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:29.695164 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e799d3-10c9-4938-aa42-8258ddb7946b admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:29.695428 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e799d3-10c9-4938-aa42-8258ddb7946b admin admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:29.696041 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e799d3-10c9-4938-aa42-8258ddb7946b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:29.698098 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e799d3-10c9-4938-aa42-8258ddb7946b admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:29.724927 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1410/2840] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 9 12:24:29 2022] POST /identity/v3/projects => generated 387 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:24:29.729115 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46a6d1f-0091-4105-9ddc-7a1c98ccf132 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:29.739226 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46a6d1f-0091-4105-9ddc-7a1c98ccf132 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:29.794757 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca27996-40d9-4482-b79c-12c7a5d09a60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:29.795312 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca27996-40d9-4482-b79c-12c7a5d09a60 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:29.795590 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca27996-40d9-4482-b79c-12c7a5d09a60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:29.795861 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca27996-40d9-4482-b79c-12c7a5d09a60 admin admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:29.796474 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca27996-40d9-4482-b79c-12c7a5d09a60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:29.798361 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca27996-40d9-4482-b79c-12c7a5d09a60 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:29.819903 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1431/2841] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:24:29 2022] POST /identity/v3/users => generated 294 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:24:29.824103 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e799d3-10c9-4938-aa42-8258ddb7946b admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:29.835091 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e799d3-10c9-4938-aa42-8258ddb7946b admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:29.839617 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc1d0a1-4031-49f4-ae65-233cb557dc7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:29.840224 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc1d0a1-4031-49f4-ae65-233cb557dc7b admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:29.840495 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc1d0a1-4031-49f4-ae65-233cb557dc7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:29.840779 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc1d0a1-4031-49f4-ae65-233cb557dc7b admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:29.841677 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc1d0a1-4031-49f4-ae65-233cb557dc7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:29.843287 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc1d0a1-4031-49f4-ae65-233cb557dc7b admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:29.852951 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1411/2842] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:24:29 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:24:29.856816 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca27996-40d9-4482-b79c-12c7a5d09a60 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:29.866482 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca27996-40d9-4482-b79c-12c7a5d09a60 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:29.871193 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f6e7e1-1985-495f-a46c-cbb21ce91ad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:29.871780 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f6e7e1-1985-495f-a46c-cbb21ce91ad2 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:29.872059 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f6e7e1-1985-495f-a46c-cbb21ce91ad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:29.873012 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f6e7e1-1985-495f-a46c-cbb21ce91ad2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:29.873656 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f6e7e1-1985-495f-a46c-cbb21ce91ad2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:29.875260 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f6e7e1-1985-495f-a46c-cbb21ce91ad2 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:29.885774 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1432/2843] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:24:29 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:24:29.890372 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc1d0a1-4031-49f4-ae65-233cb557dc7b admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:29.900053 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc1d0a1-4031-49f4-ae65-233cb557dc7b admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:29.904990 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa9fe38-2232-455f-96f6-039315e86d96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:29.905633 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa9fe38-2232-455f-96f6-039315e86d96 admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:29.905908 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa9fe38-2232-455f-96f6-039315e86d96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:29.906175 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa9fe38-2232-455f-96f6-039315e86d96 admin admin] PATH_INFO: `/v3/projects/520b4d65925340639b99d8cc2d24ca3a/users/1f41037d80574960bdd2ab0d4bcb8d9f/roles/eecd2f160dd844f5906f3d345780c501` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:29.920354 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa9fe38-2232-455f-96f6-039315e86d96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=520b4d65925340639b99d8cc2d24ca3a, user_id=1f41037d80574960bdd2ab0d4bcb8d9f, role_id=eecd2f160dd844f5906f3d345780c501)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:29.923020 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa9fe38-2232-455f-96f6-039315e86d96 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:29.936274 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1412/2844] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:24:29 2022] PUT /identity/v3/projects/520b4d65925340639b99d8cc2d24ca3a/users/1f41037d80574960bdd2ab0d4bcb8d9f/roles/eecd2f160dd844f5906f3d345780c501 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:24:29.946222 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d50f1d-ffc8-498f-83b8-ce64de0810d8 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:29.946717 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d50f1d-ffc8-498f-83b8-ce64de0810d8 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:29.946838 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d50f1d-ffc8-498f-83b8-ce64de0810d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:29.974453 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-a7d50f1d-ffc8-498f-83b8-ce64de0810d8 None None] MFA Rules not processed for user `1f41037d80574960bdd2ab0d4bcb8d9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:24:29.975689 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-a7d50f1d-ffc8-498f-83b8-ce64de0810d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:24:29.993767 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:24:30.041086 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1433/2845] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:24:29 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:24:30.046408 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa9fe38-2232-455f-96f6-039315e86d96 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:30.107276 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa9fe38-2232-455f-96f6-039315e86d96 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:30.111942 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39230db-25fd-4292-8da2-b0763b9162d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:30.112539 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39230db-25fd-4292-8da2-b0763b9162d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:30.112819 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39230db-25fd-4292-8da2-b0763b9162d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:30.113092 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39230db-25fd-4292-8da2-b0763b9162d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:30.113708 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39230db-25fd-4292-8da2-b0763b9162d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:30.115657 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39230db-25fd-4292-8da2-b0763b9162d9 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:30.141706 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1413/2846] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 9 12:24:30 2022] POST /identity/v3/projects => generated 387 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:24:30.145806 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d50f1d-ffc8-498f-83b8-ce64de0810d8 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:30.156569 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d50f1d-ffc8-498f-83b8-ce64de0810d8 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:30.213149 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ec5679-9b2e-44ba-80b5-09ab9563d813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:30.213743 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ec5679-9b2e-44ba-80b5-09ab9563d813 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:30.214017 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ec5679-9b2e-44ba-80b5-09ab9563d813 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:30.214283 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ec5679-9b2e-44ba-80b5-09ab9563d813 admin admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:30.214891 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ec5679-9b2e-44ba-80b5-09ab9563d813 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:30.217198 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ec5679-9b2e-44ba-80b5-09ab9563d813 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:30.239536 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1434/2847] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:24:30 2022] POST /identity/v3/users => generated 300 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:24:30.243686 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39230db-25fd-4292-8da2-b0763b9162d9 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:30.254735 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39230db-25fd-4292-8da2-b0763b9162d9 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:30.259372 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b2e8a8-e128-4af0-9112-33b36bc4c4ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:30.259963 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b2e8a8-e128-4af0-9112-33b36bc4c4ff admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:30.260239 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b2e8a8-e128-4af0-9112-33b36bc4c4ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:30.260514 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b2e8a8-e128-4af0-9112-33b36bc4c4ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:30.261144 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b2e8a8-e128-4af0-9112-33b36bc4c4ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:30.262738 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b2e8a8-e128-4af0-9112-33b36bc4c4ff admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:30.273324 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1414/2848] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:24:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:24:30.277377 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ec5679-9b2e-44ba-80b5-09ab9563d813 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:30.287432 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ec5679-9b2e-44ba-80b5-09ab9563d813 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:30.292003 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f823fb-f292-46eb-981b-85cd55e6be80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:30.292762 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f823fb-f292-46eb-981b-85cd55e6be80 admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:30.293054 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f823fb-f292-46eb-981b-85cd55e6be80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:30.293326 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f823fb-f292-46eb-981b-85cd55e6be80 admin admin] PATH_INFO: `/v3/projects/38e3e416b0044c939b8d567556876e3a/users/1b19f15e89d443bba74b21d7a0b5ac2f/roles/3f2ca33902a04b6dbc2ef7d06d7289d3` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:30.308674 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f823fb-f292-46eb-981b-85cd55e6be80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38e3e416b0044c939b8d567556876e3a, user_id=1b19f15e89d443bba74b21d7a0b5ac2f, role_id=3f2ca33902a04b6dbc2ef7d06d7289d3)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:30.311831 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f823fb-f292-46eb-981b-85cd55e6be80 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:30.324331 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1435/2849] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:24:30 2022] PUT /identity/v3/projects/38e3e416b0044c939b8d567556876e3a/users/1b19f15e89d443bba74b21d7a0b5ac2f/roles/3f2ca33902a04b6dbc2ef7d06d7289d3 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:24:30.328475 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b2e8a8-e128-4af0-9112-33b36bc4c4ff admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:30.338461 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b2e8a8-e128-4af0-9112-33b36bc4c4ff admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:30.395578 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceebbc90-c7f2-4b64-9f32-f1aafcb3f122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:30.396211 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceebbc90-c7f2-4b64-9f32-f1aafcb3f122 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:30.396492 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceebbc90-c7f2-4b64-9f32-f1aafcb3f122 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:30.396765 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceebbc90-c7f2-4b64-9f32-f1aafcb3f122 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:30.397395 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceebbc90-c7f2-4b64-9f32-f1aafcb3f122 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:30.399021 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceebbc90-c7f2-4b64-9f32-f1aafcb3f122 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:30.409302 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1415/2850] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:24:30 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:24:30.413369 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f823fb-f292-46eb-981b-85cd55e6be80 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:30.424485 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f823fb-f292-46eb-981b-85cd55e6be80 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:30.429141 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e958393-9b3e-410f-abb3-87fd62ef41d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:30.429785 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e958393-9b3e-410f-abb3-87fd62ef41d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:30.430065 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e958393-9b3e-410f-abb3-87fd62ef41d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:30.430339 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e958393-9b3e-410f-abb3-87fd62ef41d5 admin admin] PATH_INFO: `/v3/projects/38e3e416b0044c939b8d567556876e3a/users/1b19f15e89d443bba74b21d7a0b5ac2f/roles/3f2ca33902a04b6dbc2ef7d06d7289d3` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:30.432299 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e958393-9b3e-410f-abb3-87fd62ef41d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38e3e416b0044c939b8d567556876e3a, user_id=1b19f15e89d443bba74b21d7a0b5ac2f, role_id=3f2ca33902a04b6dbc2ef7d06d7289d3)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:30.434997 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e958393-9b3e-410f-abb3-87fd62ef41d5 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:30.444962 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1436/2851] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:24:30 2022] PUT /identity/v3/projects/38e3e416b0044c939b8d567556876e3a/users/1b19f15e89d443bba74b21d7a0b5ac2f/roles/3f2ca33902a04b6dbc2ef7d06d7289d3 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:24:30.455191 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bcc398-63e4-46e1-8ade-f13c3f54d587 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:30.455695 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bcc398-63e4-46e1-8ade-f13c3f54d587 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:30.455796 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bcc398-63e4-46e1-8ade-f13c3f54d587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:30.480952 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:24:30.484753 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-32bcc398-63e4-46e1-8ade-f13c3f54d587 None None] MFA Rules not processed for user `1b19f15e89d443bba74b21d7a0b5ac2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:24:30.486629 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-32bcc398-63e4-46e1-8ade-f13c3f54d587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:24:30.553477 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1416/2852] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:24:30 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:24:30.564327 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e958393-9b3e-410f-abb3-87fd62ef41d5 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:30.565993 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-1e958393-9b3e-410f-abb3-87fd62ef41d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:24:30.623001 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e958393-9b3e-410f-abb3-87fd62ef41d5 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:30.627569 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf08167-a592-4c95-bc1c-ff10f47af90f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535c258100df40b58a5ee44b2c21d621', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:30.628245 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf08167-a592-4c95-bc1c-ff10f47af90f service cinder] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:30.628631 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf08167-a592-4c95-bc1c-ff10f47af90f service cinder] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:30.628901 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf08167-a592-4c95-bc1c-ff10f47af90f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:30.638567 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf08167-a592-4c95-bc1c-ff10f47af90f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:30.640085 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf08167-a592-4c95-bc1c-ff10f47af90f service cinder] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:30.649617 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1437/2853] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:24:30 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:24:31.184918 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bcc398-63e4-46e1-8ade-f13c3f54d587 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:31.247236 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bcc398-63e4-46e1-8ade-f13c3f54d587 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:31.251772 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dca930-8ebb-4752-b674-d8dbba761d32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:31.252424 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dca930-8ebb-4752-b674-d8dbba761d32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:31.252701 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dca930-8ebb-4752-b674-d8dbba761d32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:31.252971 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dca930-8ebb-4752-b674-d8dbba761d32 admin admin] PATH_INFO: `/v3/users/1f41037d80574960bdd2ab0d4bcb8d9f` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:31.254096 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dca930-8ebb-4752-b674-d8dbba761d32 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f41037d80574960bdd2ab0d4bcb8d9f)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:31.256064 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dca930-8ebb-4752-b674-d8dbba761d32 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:31.302037 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-d7dca930-8ebb-4752-b674-d8dbba761d32 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f41037d80574960bdd2ab0d4bcb8d9f'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:31.314225 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-d7dca930-8ebb-4752-b674-d8dbba761d32 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f41037d80574960bdd2ab0d4bcb8d9f'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:31.321071 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-d7dca930-8ebb-4752-b674-d8dbba761d32 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f41037d80574960bdd2ab0d4bcb8d9f'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:31.328015 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-d7dca930-8ebb-4752-b674-d8dbba761d32 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f41037d80574960bdd2ab0d4bcb8d9f'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:31.328753 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-d7dca930-8ebb-4752-b674-d8dbba761d32 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f41037d80574960bdd2ab0d4bcb8d9f'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:31.334582 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1417/2854] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 9 12:24:31 2022] DELETE /identity/v3/users/1f41037d80574960bdd2ab0d4bcb8d9f => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:24:31.339251 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b435add-0c18-4130-abd3-88f61e1e96be None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:31.339739 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b435add-0c18-4130-abd3-88f61e1e96be None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:31.339739 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b435add-0c18-4130-abd3-88f61e1e96be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:31.359610 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-1b435add-0c18-4130-abd3-88f61e1e96be None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:24:31.360792 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-1b435add-0c18-4130-abd3-88f61e1e96be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:24:31.369645 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:24:31.422483 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1438/2855] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:24:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:24:31.432505 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dca930-8ebb-4752-b674-d8dbba761d32 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:31.433901 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-d7dca930-8ebb-4752-b674-d8dbba761d32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:24:31.454260 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:24:31.493291 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dca930-8ebb-4752-b674-d8dbba761d32 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:31.496912 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85808ce5-dc0e-446f-9d23-b53940e7babe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:31.497701 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85808ce5-dc0e-446f-9d23-b53940e7babe service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:31.498092 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85808ce5-dc0e-446f-9d23-b53940e7babe service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:31.498386 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85808ce5-dc0e-446f-9d23-b53940e7babe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:31.507369 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85808ce5-dc0e-446f-9d23-b53940e7babe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:31.508785 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85808ce5-dc0e-446f-9d23-b53940e7babe service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:31.519318 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1418/2856] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:24:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:24:31.790025 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b435add-0c18-4130-abd3-88f61e1e96be None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:31.791564 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-1b435add-0c18-4130-abd3-88f61e1e96be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:24:31.804684 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b435add-0c18-4130-abd3-88f61e1e96be None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:31.809836 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d942aaf5-86f8-4fff-bbc0-1c2e8c115fac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:31.810788 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d942aaf5-86f8-4fff-bbc0-1c2e8c115fac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:31.811106 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d942aaf5-86f8-4fff-bbc0-1c2e8c115fac admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:31.811356 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d942aaf5-86f8-4fff-bbc0-1c2e8c115fac admin admin] PATH_INFO: `/v3/projects/520b4d65925340639b99d8cc2d24ca3a` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:31.812484 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d942aaf5-86f8-4fff-bbc0-1c2e8c115fac admin admin] RBAC: Authorizing `identity:delete_project(project_id=520b4d65925340639b99d8cc2d24ca3a)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:31.814382 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d942aaf5-86f8-4fff-bbc0-1c2e8c115fac admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:31.863616 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-d942aaf5-86f8-4fff-bbc0-1c2e8c115fac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '520b4d65925340639b99d8cc2d24ca3a', 'request_id': 'req-d942aaf5-86f8-4fff-bbc0-1c2e8c115fac'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:31.868973 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-d942aaf5-86f8-4fff-bbc0-1c2e8c115fac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '520b4d65925340639b99d8cc2d24ca3a', 'request_id': 'req-d942aaf5-86f8-4fff-bbc0-1c2e8c115fac'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:31.875631 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-d942aaf5-86f8-4fff-bbc0-1c2e8c115fac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '520b4d65925340639b99d8cc2d24ca3a', 'request_id': 'req-d942aaf5-86f8-4fff-bbc0-1c2e8c115fac'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:31.887766 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-d942aaf5-86f8-4fff-bbc0-1c2e8c115fac admin admin] The token cache is being invalidate because project 520b4d65925340639b99d8cc2d24ca3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81591) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:24:31.888639 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-d942aaf5-86f8-4fff-bbc0-1c2e8c115fac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:31.889393 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-d942aaf5-86f8-4fff-bbc0-1c2e8c115fac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:31.890867 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1439/2857] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 9 12:24:31 2022] DELETE /identity/v3/projects/520b4d65925340639b99d8cc2d24ca3a => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:24:31.894791 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85808ce5-dc0e-446f-9d23-b53940e7babe service neutron] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:31.904546 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85808ce5-dc0e-446f-9d23-b53940e7babe service neutron] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:31.959960 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53720cd4-5e0e-4b4c-9566-74ae395e9adf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:31.960583 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53720cd4-5e0e-4b4c-9566-74ae395e9adf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:31.960860 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53720cd4-5e0e-4b4c-9566-74ae395e9adf admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:31.961129 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53720cd4-5e0e-4b4c-9566-74ae395e9adf admin admin] PATH_INFO: `/v3/users/1b19f15e89d443bba74b21d7a0b5ac2f` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:31.962234 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53720cd4-5e0e-4b4c-9566-74ae395e9adf admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b19f15e89d443bba74b21d7a0b5ac2f)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:31.964176 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53720cd4-5e0e-4b4c-9566-74ae395e9adf admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:32.008815 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-53720cd4-5e0e-4b4c-9566-74ae395e9adf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b19f15e89d443bba74b21d7a0b5ac2f'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:32.027944 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-53720cd4-5e0e-4b4c-9566-74ae395e9adf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b19f15e89d443bba74b21d7a0b5ac2f'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:32.035076 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-53720cd4-5e0e-4b4c-9566-74ae395e9adf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b19f15e89d443bba74b21d7a0b5ac2f'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:32.042355 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-53720cd4-5e0e-4b4c-9566-74ae395e9adf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b19f15e89d443bba74b21d7a0b5ac2f'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:32.042988 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-53720cd4-5e0e-4b4c-9566-74ae395e9adf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b19f15e89d443bba74b21d7a0b5ac2f'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:32.049169 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1419/2858] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 9 12:24:31 2022] DELETE /identity/v3/users/1b19f15e89d443bba74b21d7a0b5ac2f => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:24:32.321561 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d942aaf5-86f8-4fff-bbc0-1c2e8c115fac admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:32.323121 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-d942aaf5-86f8-4fff-bbc0-1c2e8c115fac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:24:32.337096 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d942aaf5-86f8-4fff-bbc0-1c2e8c115fac admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:32.392794 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f7114c-eb4f-402c-87fd-96f8f1929c36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:32.393506 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f7114c-eb4f-402c-87fd-96f8f1929c36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:32.393727 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f7114c-eb4f-402c-87fd-96f8f1929c36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:32.393995 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f7114c-eb4f-402c-87fd-96f8f1929c36 admin admin] PATH_INFO: `/v3/projects/38e3e416b0044c939b8d567556876e3a` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:32.395130 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f7114c-eb4f-402c-87fd-96f8f1929c36 admin admin] RBAC: Authorizing `identity:delete_project(project_id=38e3e416b0044c939b8d567556876e3a)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:32.397025 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f7114c-eb4f-402c-87fd-96f8f1929c36 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:32.437248 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:24:32.444451 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-63f7114c-eb4f-402c-87fd-96f8f1929c36 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38e3e416b0044c939b8d567556876e3a', 'request_id': 'req-63f7114c-eb4f-402c-87fd-96f8f1929c36'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:32.450546 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-63f7114c-eb4f-402c-87fd-96f8f1929c36 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38e3e416b0044c939b8d567556876e3a', 'request_id': 'req-63f7114c-eb4f-402c-87fd-96f8f1929c36'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:32.456478 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-63f7114c-eb4f-402c-87fd-96f8f1929c36 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38e3e416b0044c939b8d567556876e3a', 'request_id': 'req-63f7114c-eb4f-402c-87fd-96f8f1929c36'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:32.468304 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-63f7114c-eb4f-402c-87fd-96f8f1929c36 admin admin] The token cache is being invalidate because project 38e3e416b0044c939b8d567556876e3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81591) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:24:32.468595 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-63f7114c-eb4f-402c-87fd-96f8f1929c36 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:32.469800 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-63f7114c-eb4f-402c-87fd-96f8f1929c36 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:32.471305 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1440/2859] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 9 12:24:32 2022] DELETE /identity/v3/projects/38e3e416b0044c939b8d567556876e3a => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:24:32.476158 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433baeb5-25da-4fbf-a91c-df9602c9b29e None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:32.476437 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433baeb5-25da-4fbf-a91c-df9602c9b29e None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:32.476704 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433baeb5-25da-4fbf-a91c-df9602c9b29e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:32.496721 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-433baeb5-25da-4fbf-a91c-df9602c9b29e None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:24:32.499017 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-433baeb5-25da-4fbf-a91c-df9602c9b29e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:24:32.532471 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:24:32.561144 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1420/2860] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:24:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:24:32.624830 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19d0674-e072-4a40-ae42-236d186bc1c4 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:32.625202 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19d0674-e072-4a40-ae42-236d186bc1c4 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:32.625483 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19d0674-e072-4a40-ae42-236d186bc1c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:32.645633 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-d19d0674-e072-4a40-ae42-236d186bc1c4 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:24:32.646876 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-d19d0674-e072-4a40-ae42-236d186bc1c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:24:32.655258 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1441/2861] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:24:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:24:32.659635 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433baeb5-25da-4fbf-a91c-df9602c9b29e None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:32.669889 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433baeb5-25da-4fbf-a91c-df9602c9b29e None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:32.674806 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fddb143-0d9b-43ff-afe5-a5b204a183a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:32.675434 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fddb143-0d9b-43ff-afe5-a5b204a183a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:32.675712 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fddb143-0d9b-43ff-afe5-a5b204a183a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:32.675980 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fddb143-0d9b-43ff-afe5-a5b204a183a4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:32.676619 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fddb143-0d9b-43ff-afe5-a5b204a183a4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:24:32.676916 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fddb143-0d9b-43ff-afe5-a5b204a183a4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:32.678505 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fddb143-0d9b-43ff-afe5-a5b204a183a4 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:32.690160 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1421/2862] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 9 12:24:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:24:32.694486 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19d0674-e072-4a40-ae42-236d186bc1c4 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:32.703987 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19d0674-e072-4a40-ae42-236d186bc1c4 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:32.708722 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e87873e-2c4b-4fd8-8569-26b4eb5ffa44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:32.709369 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e87873e-2c4b-4fd8-8569-26b4eb5ffa44 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:32.709649 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e87873e-2c4b-4fd8-8569-26b4eb5ffa44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:32.709922 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e87873e-2c4b-4fd8-8569-26b4eb5ffa44 admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:32.710575 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e87873e-2c4b-4fd8-8569-26b4eb5ffa44 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:32.712506 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e87873e-2c4b-4fd8-8569-26b4eb5ffa44 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:32.740605 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1442/2863] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 9 12:24:32 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:24:32.744602 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fddb143-0d9b-43ff-afe5-a5b204a183a4 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:32.755607 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fddb143-0d9b-43ff-afe5-a5b204a183a4 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:32.811500 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb9c103-fd6a-47e3-aa9b-ab84b1407c1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:32.812129 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb9c103-fd6a-47e3-aa9b-ab84b1407c1f admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:32.812409 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb9c103-fd6a-47e3-aa9b-ab84b1407c1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:32.812678 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb9c103-fd6a-47e3-aa9b-ab84b1407c1f admin admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:32.813296 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb9c103-fd6a-47e3-aa9b-ab84b1407c1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:32.815229 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb9c103-fd6a-47e3-aa9b-ab84b1407c1f admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:32.837205 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1422/2864] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:24:32 2022] POST /identity/v3/users => generated 292 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:24:32.842136 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e87873e-2c4b-4fd8-8569-26b4eb5ffa44 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:32.852241 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e87873e-2c4b-4fd8-8569-26b4eb5ffa44 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:32.857530 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb1c82a-490d-4677-b243-53ad55695974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:32.858289 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb1c82a-490d-4677-b243-53ad55695974 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:32.858561 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb1c82a-490d-4677-b243-53ad55695974 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:32.858827 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb1c82a-490d-4677-b243-53ad55695974 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:32.859495 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb1c82a-490d-4677-b243-53ad55695974 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:32.861071 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb1c82a-490d-4677-b243-53ad55695974 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:32.870748 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1443/2865] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:24:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:24:32.874679 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb9c103-fd6a-47e3-aa9b-ab84b1407c1f admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:32.884556 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb9c103-fd6a-47e3-aa9b-ab84b1407c1f admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:32.889334 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66932024-8fa5-4dbf-9808-7a0c9d38149f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:32.889921 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66932024-8fa5-4dbf-9808-7a0c9d38149f admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:32.890196 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66932024-8fa5-4dbf-9808-7a0c9d38149f admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:32.890462 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66932024-8fa5-4dbf-9808-7a0c9d38149f admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:32.891145 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66932024-8fa5-4dbf-9808-7a0c9d38149f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:32.892727 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66932024-8fa5-4dbf-9808-7a0c9d38149f admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:32.903138 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1423/2866] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:24:32 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:24:32.907973 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb1c82a-490d-4677-b243-53ad55695974 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:32.917657 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb1c82a-490d-4677-b243-53ad55695974 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:32.923108 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6041e027-06ce-4989-9d5b-dd11d73c0487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:32.924004 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6041e027-06ce-4989-9d5b-dd11d73c0487 admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:32.924279 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6041e027-06ce-4989-9d5b-dd11d73c0487 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:32.924546 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6041e027-06ce-4989-9d5b-dd11d73c0487 admin admin] PATH_INFO: `/v3/projects/358444869b9d408bb15d9dd69220c022/users/81b0b8db5c524eadb8f636bb41ae6085/roles/eecd2f160dd844f5906f3d345780c501` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:32.939284 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6041e027-06ce-4989-9d5b-dd11d73c0487 admin admin] RBAC: Authorizing `identity:create_grant(project_id=358444869b9d408bb15d9dd69220c022, user_id=81b0b8db5c524eadb8f636bb41ae6085, role_id=eecd2f160dd844f5906f3d345780c501)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:32.941901 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6041e027-06ce-4989-9d5b-dd11d73c0487 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:32.954262 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1444/2867] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:24:32 2022] PUT /identity/v3/projects/358444869b9d408bb15d9dd69220c022/users/81b0b8db5c524eadb8f636bb41ae6085/roles/eecd2f160dd844f5906f3d345780c501 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:24:32.964201 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11f573c-7773-47c0-95c6-25d1f54a97cd None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:32.964484 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11f573c-7773-47c0-95c6-25d1f54a97cd None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:32.964752 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11f573c-7773-47c0-95c6-25d1f54a97cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:32.992776 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-e11f573c-7773-47c0-95c6-25d1f54a97cd None None] MFA Rules not processed for user `81b0b8db5c524eadb8f636bb41ae6085`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:24:32.993978 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-e11f573c-7773-47c0-95c6-25d1f54a97cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:24:33.057047 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1424/2868] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:24:32 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:24:33.067936 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6041e027-06ce-4989-9d5b-dd11d73c0487 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:33.069596 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-6041e027-06ce-4989-9d5b-dd11d73c0487 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:24:33.127022 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6041e027-06ce-4989-9d5b-dd11d73c0487 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:33.130636 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb9ce18-2d45-42b4-a24f-794b115aca43 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535c258100df40b58a5ee44b2c21d621', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:33.131269 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb9ce18-2d45-42b4-a24f-794b115aca43 service cinder] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:33.131552 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb9ce18-2d45-42b4-a24f-794b115aca43 service cinder] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:33.131826 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb9ce18-2d45-42b4-a24f-794b115aca43 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:33.140757 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb9ce18-2d45-42b4-a24f-794b115aca43 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:33.142177 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb9ce18-2d45-42b4-a24f-794b115aca43 service cinder] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:33.150749 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1445/2869] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:24:33 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:24:40.699550 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11f573c-7773-47c0-95c6-25d1f54a97cd None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:40.738100 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:24:40.764179 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11f573c-7773-47c0-95c6-25d1f54a97cd None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:40.768593 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60088125-2bdd-42ca-88a3-1f5253e7746f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:40.769213 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60088125-2bdd-42ca-88a3-1f5253e7746f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:40.769484 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60088125-2bdd-42ca-88a3-1f5253e7746f admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:40.769749 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60088125-2bdd-42ca-88a3-1f5253e7746f admin admin] PATH_INFO: `/v3/users/81b0b8db5c524eadb8f636bb41ae6085` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:40.770848 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60088125-2bdd-42ca-88a3-1f5253e7746f admin admin] RBAC: Authorizing `identity:delete_user(user_id=81b0b8db5c524eadb8f636bb41ae6085)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:40.772779 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60088125-2bdd-42ca-88a3-1f5253e7746f admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:40.818593 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-60088125-2bdd-42ca-88a3-1f5253e7746f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81b0b8db5c524eadb8f636bb41ae6085'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:40.830838 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-60088125-2bdd-42ca-88a3-1f5253e7746f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81b0b8db5c524eadb8f636bb41ae6085'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:40.837666 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-60088125-2bdd-42ca-88a3-1f5253e7746f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81b0b8db5c524eadb8f636bb41ae6085'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:40.845143 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-60088125-2bdd-42ca-88a3-1f5253e7746f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81b0b8db5c524eadb8f636bb41ae6085'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:40.845764 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-60088125-2bdd-42ca-88a3-1f5253e7746f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81b0b8db5c524eadb8f636bb41ae6085'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:40.851623 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1425/2870] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 9 12:24:40 2022] DELETE /identity/v3/users/81b0b8db5c524eadb8f636bb41ae6085 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:24:40.856275 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013e29c2-73d5-4cff-8300-0c0469fb0d57 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:40.857343 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013e29c2-73d5-4cff-8300-0c0469fb0d57 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:40.857512 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013e29c2-73d5-4cff-8300-0c0469fb0d57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:40.876117 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-013e29c2-73d5-4cff-8300-0c0469fb0d57 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:24:40.877323 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-013e29c2-73d5-4cff-8300-0c0469fb0d57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:24:40.934893 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1446/2871] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:24:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:24:40.944628 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60088125-2bdd-42ca-88a3-1f5253e7746f admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:40.946017 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-60088125-2bdd-42ca-88a3-1f5253e7746f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:24:41.003240 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60088125-2bdd-42ca-88a3-1f5253e7746f admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:41.007154 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78df56c7-f4c6-47f6-8790-15e1412bfb49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:41.007753 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78df56c7-f4c6-47f6-8790-15e1412bfb49 service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:41.008032 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78df56c7-f4c6-47f6-8790-15e1412bfb49 service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:41.008303 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78df56c7-f4c6-47f6-8790-15e1412bfb49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:41.017489 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78df56c7-f4c6-47f6-8790-15e1412bfb49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:41.018946 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78df56c7-f4c6-47f6-8790-15e1412bfb49 service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:41.028361 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1426/2872] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:24:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:24:41.297996 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013e29c2-73d5-4cff-8300-0c0469fb0d57 None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:41.299747 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-013e29c2-73d5-4cff-8300-0c0469fb0d57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:24:41.312179 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013e29c2-73d5-4cff-8300-0c0469fb0d57 None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:41.317076 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41eb357-3561-40f2-834f-aa4604d48de0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:41.317818 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41eb357-3561-40f2-834f-aa4604d48de0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:41.318170 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41eb357-3561-40f2-834f-aa4604d48de0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:41.318451 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41eb357-3561-40f2-834f-aa4604d48de0 admin admin] PATH_INFO: `/v3/projects/358444869b9d408bb15d9dd69220c022` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:41.320308 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41eb357-3561-40f2-834f-aa4604d48de0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=358444869b9d408bb15d9dd69220c022)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:41.322278 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41eb357-3561-40f2-834f-aa4604d48de0 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:41.324680 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:24:41.373515 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-c41eb357-3561-40f2-834f-aa4604d48de0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '358444869b9d408bb15d9dd69220c022', 'request_id': 'req-c41eb357-3561-40f2-834f-aa4604d48de0'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:41.379264 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-c41eb357-3561-40f2-834f-aa4604d48de0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '358444869b9d408bb15d9dd69220c022', 'request_id': 'req-c41eb357-3561-40f2-834f-aa4604d48de0'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:41.385468 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-c41eb357-3561-40f2-834f-aa4604d48de0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '358444869b9d408bb15d9dd69220c022', 'request_id': 'req-c41eb357-3561-40f2-834f-aa4604d48de0'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:41.399069 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-c41eb357-3561-40f2-834f-aa4604d48de0 admin admin] The token cache is being invalidate because project 358444869b9d408bb15d9dd69220c022 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81591) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:24:41.399358 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-c41eb357-3561-40f2-834f-aa4604d48de0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:41.399979 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-c41eb357-3561-40f2-834f-aa4604d48de0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:41.401456 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1447/2873] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 9 12:24:41 2022] DELETE /identity/v3/projects/358444869b9d408bb15d9dd69220c022 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:24:41.407151 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f88a19-8fa5-4e35-a62e-8a568ed719c1 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:41.407438 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f88a19-8fa5-4e35-a62e-8a568ed719c1 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:41.407710 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f88a19-8fa5-4e35-a62e-8a568ed719c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:41.428363 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-b6f88a19-8fa5-4e35-a62e-8a568ed719c1 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:24:41.429615 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-b6f88a19-8fa5-4e35-a62e-8a568ed719c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:24:41.479133 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:24:41.489518 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1427/2874] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:24:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:24:41.498689 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133e6578-26c0-4b39-a2c8-b1e439d9276e None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:41.499008 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133e6578-26c0-4b39-a2c8-b1e439d9276e None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:41.499288 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133e6578-26c0-4b39-a2c8-b1e439d9276e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:41.519209 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-133e6578-26c0-4b39-a2c8-b1e439d9276e None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:24:41.520415 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-133e6578-26c0-4b39-a2c8-b1e439d9276e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:24:41.528475 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1448/2875] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:24:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:24:41.533304 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f88a19-8fa5-4e35-a62e-8a568ed719c1 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:41.542808 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f88a19-8fa5-4e35-a62e-8a568ed719c1 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:41.547751 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7b88ec-8e1d-4120-8d48-e0902b4833e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:41.548377 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7b88ec-8e1d-4120-8d48-e0902b4833e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:41.548655 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7b88ec-8e1d-4120-8d48-e0902b4833e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:41.548926 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7b88ec-8e1d-4120-8d48-e0902b4833e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:41.549834 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7b88ec-8e1d-4120-8d48-e0902b4833e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81590) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:24:41.550132 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7b88ec-8e1d-4120-8d48-e0902b4833e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:41.551764 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7b88ec-8e1d-4120-8d48-e0902b4833e3 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:41.563924 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1428/2876] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 9 12:24:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 12:24:41.568129 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133e6578-26c0-4b39-a2c8-b1e439d9276e None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:41.578037 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133e6578-26c0-4b39-a2c8-b1e439d9276e None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:41.582742 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b391e69-4773-4394-bbea-d54a5a4709d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:41.583383 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b391e69-4773-4394-bbea-d54a5a4709d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:41.583663 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b391e69-4773-4394-bbea-d54a5a4709d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:41.583929 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b391e69-4773-4394-bbea-d54a5a4709d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:41.584550 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b391e69-4773-4394-bbea-d54a5a4709d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:41.586435 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b391e69-4773-4394-bbea-d54a5a4709d4 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:41.614238 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1449/2877] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 9 12:24:41 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:24:41.618546 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7b88ec-8e1d-4120-8d48-e0902b4833e3 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:41.629120 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7b88ec-8e1d-4120-8d48-e0902b4833e3 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:41.685726 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be25589f-0a21-40e0-83f9-cc5bb379a252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:41.686329 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be25589f-0a21-40e0-83f9-cc5bb379a252 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:41.686606 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be25589f-0a21-40e0-83f9-cc5bb379a252 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:41.686875 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be25589f-0a21-40e0-83f9-cc5bb379a252 admin admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:41.687528 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be25589f-0a21-40e0-83f9-cc5bb379a252 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:41.689444 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be25589f-0a21-40e0-83f9-cc5bb379a252 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:41.710630 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1429/2878] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:24:41 2022] POST /identity/v3/users => generated 292 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:24:41.714995 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b391e69-4773-4394-bbea-d54a5a4709d4 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:41.725165 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b391e69-4773-4394-bbea-d54a5a4709d4 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:41.729962 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba2d6b1-be67-46a1-90f2-da231a81b318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:41.730503 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba2d6b1-be67-46a1-90f2-da231a81b318 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:41.730781 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba2d6b1-be67-46a1-90f2-da231a81b318 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:41.731091 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba2d6b1-be67-46a1-90f2-da231a81b318 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:41.731732 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba2d6b1-be67-46a1-90f2-da231a81b318 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:41.733366 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba2d6b1-be67-46a1-90f2-da231a81b318 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:41.743426 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1450/2879] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:24:41 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:24:41.748169 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be25589f-0a21-40e0-83f9-cc5bb379a252 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:41.757900 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be25589f-0a21-40e0-83f9-cc5bb379a252 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:41.762445 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd57a7f-be5f-4ff9-aec0-6e38ea6ceca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:41.763744 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd57a7f-be5f-4ff9-aec0-6e38ea6ceca6 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:41.764086 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd57a7f-be5f-4ff9-aec0-6e38ea6ceca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:41.764291 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd57a7f-be5f-4ff9-aec0-6e38ea6ceca6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:41.764926 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd57a7f-be5f-4ff9-aec0-6e38ea6ceca6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:41.766500 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd57a7f-be5f-4ff9-aec0-6e38ea6ceca6 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:41.776836 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1430/2880] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:24:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 12:24:41.781381 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba2d6b1-be67-46a1-90f2-da231a81b318 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:41.791229 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba2d6b1-be67-46a1-90f2-da231a81b318 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:41.796159 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6e3d5e-b3e9-42b5-ad24-b07ab5689c94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:41.796800 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6e3d5e-b3e9-42b5-ad24-b07ab5689c94 admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:41.797084 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6e3d5e-b3e9-42b5-ad24-b07ab5689c94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:41.797357 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6e3d5e-b3e9-42b5-ad24-b07ab5689c94 admin admin] PATH_INFO: `/v3/projects/34ba228e291d49fe8f431661207976eb/users/cb766a30bf634ad9ad51253e417a77a2/roles/eecd2f160dd844f5906f3d345780c501` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:41.813522 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6e3d5e-b3e9-42b5-ad24-b07ab5689c94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34ba228e291d49fe8f431661207976eb, user_id=cb766a30bf634ad9ad51253e417a77a2, role_id=eecd2f160dd844f5906f3d345780c501)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:41.816255 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6e3d5e-b3e9-42b5-ad24-b07ab5689c94 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:41.829504 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1451/2881] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:24:41 2022] PUT /identity/v3/projects/34ba228e291d49fe8f431661207976eb/users/cb766a30bf634ad9ad51253e417a77a2/roles/eecd2f160dd844f5906f3d345780c501 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:24:41.839761 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d00cfa-71f7-4432-a597-3db97a4c7754 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:41.840057 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d00cfa-71f7-4432-a597-3db97a4c7754 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:41.840316 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d00cfa-71f7-4432-a597-3db97a4c7754 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:41.867276 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-b0d00cfa-71f7-4432-a597-3db97a4c7754 None None] MFA Rules not processed for user `cb766a30bf634ad9ad51253e417a77a2`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:24:41.868467 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-b0d00cfa-71f7-4432-a597-3db97a4c7754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:24:41.930992 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1431/2882] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:24:41 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:24:41.935989 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6e3d5e-b3e9-42b5-ad24-b07ab5689c94 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:41.996563 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6e3d5e-b3e9-42b5-ad24-b07ab5689c94 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:42.001811 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e554fe32-6b11-44d1-bf66-d9ea128faf48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:42.002404 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e554fe32-6b11-44d1-bf66-d9ea128faf48 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:42.002680 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e554fe32-6b11-44d1-bf66-d9ea128faf48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:42.002978 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e554fe32-6b11-44d1-bf66-d9ea128faf48 admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:42.003612 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e554fe32-6b11-44d1-bf66-d9ea128faf48 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:42.005490 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e554fe32-6b11-44d1-bf66-d9ea128faf48 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:42.031178 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1452/2883] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 9 12:24:41 2022] POST /identity/v3/projects => generated 383 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:24:42.035623 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d00cfa-71f7-4432-a597-3db97a4c7754 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:42.048125 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d00cfa-71f7-4432-a597-3db97a4c7754 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:42.095931 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:24:42.105453 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c4129c-23e5-4225-b8bc-395965a9b666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:42.106056 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c4129c-23e5-4225-b8bc-395965a9b666 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:42.106332 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c4129c-23e5-4225-b8bc-395965a9b666 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:42.106623 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c4129c-23e5-4225-b8bc-395965a9b666 admin admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:42.107787 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c4129c-23e5-4225-b8bc-395965a9b666 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:42.109700 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c4129c-23e5-4225-b8bc-395965a9b666 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:42.130672 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1432/2884] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:24:42 2022] POST /identity/v3/users => generated 292 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:24:42.134703 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e554fe32-6b11-44d1-bf66-d9ea128faf48 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:42.144908 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e554fe32-6b11-44d1-bf66-d9ea128faf48 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:42.149572 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dc7dde-5da2-4c54-b7d6-cbbeaad46359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:42.150167 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dc7dde-5da2-4c54-b7d6-cbbeaad46359 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:42.150447 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dc7dde-5da2-4c54-b7d6-cbbeaad46359 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:42.150719 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dc7dde-5da2-4c54-b7d6-cbbeaad46359 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:42.151406 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dc7dde-5da2-4c54-b7d6-cbbeaad46359 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:42.153007 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dc7dde-5da2-4c54-b7d6-cbbeaad46359 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:42.163524 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1453/2885] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:24:42 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:24:42.167410 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c4129c-23e5-4225-b8bc-395965a9b666 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:42.178190 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c4129c-23e5-4225-b8bc-395965a9b666 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:42.182765 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa7cfc3-89bb-48b8-b8f5-af584879c257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:42.183386 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa7cfc3-89bb-48b8-b8f5-af584879c257 admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:42.183665 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa7cfc3-89bb-48b8-b8f5-af584879c257 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:42.183933 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa7cfc3-89bb-48b8-b8f5-af584879c257 admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:42.184561 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa7cfc3-89bb-48b8-b8f5-af584879c257 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:42.186726 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa7cfc3-89bb-48b8-b8f5-af584879c257 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:42.196594 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1433/2886] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:24:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:24:42.200484 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dc7dde-5da2-4c54-b7d6-cbbeaad46359 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:42.205618 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:24:42.212499 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dc7dde-5da2-4c54-b7d6-cbbeaad46359 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:42.217911 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22178de0-d5d3-4d5d-b1bc-d6d9ce3e795b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:42.218583 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22178de0-d5d3-4d5d-b1bc-d6d9ce3e795b admin admin] REQUEST_METHOD: `PUT` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:42.218890 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22178de0-d5d3-4d5d-b1bc-d6d9ce3e795b admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:42.219192 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22178de0-d5d3-4d5d-b1bc-d6d9ce3e795b admin admin] PATH_INFO: `/v3/projects/6d428fb6c1f84ba8b0974c1b1609b62a/users/75fde45dcf054c808a484902db439191/roles/eecd2f160dd844f5906f3d345780c501` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:42.234719 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22178de0-d5d3-4d5d-b1bc-d6d9ce3e795b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d428fb6c1f84ba8b0974c1b1609b62a, user_id=75fde45dcf054c808a484902db439191, role_id=eecd2f160dd844f5906f3d345780c501)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:42.237455 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22178de0-d5d3-4d5d-b1bc-d6d9ce3e795b admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:42.250356 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1454/2887] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:24:42 2022] PUT /identity/v3/projects/6d428fb6c1f84ba8b0974c1b1609b62a/users/75fde45dcf054c808a484902db439191/roles/eecd2f160dd844f5906f3d345780c501 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:24:42.260752 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c909a08f-9a48-4f7f-8ad0-3ab7f758955a None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:42.261034 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c909a08f-9a48-4f7f-8ad0-3ab7f758955a None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:42.261303 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c909a08f-9a48-4f7f-8ad0-3ab7f758955a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:42.288237 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-c909a08f-9a48-4f7f-8ad0-3ab7f758955a None None] MFA Rules not processed for user `75fde45dcf054c808a484902db439191`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:24:42.289447 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-c909a08f-9a48-4f7f-8ad0-3ab7f758955a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:24:42.354029 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1434/2888] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:24:42 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:24:42.359361 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22178de0-d5d3-4d5d-b1bc-d6d9ce3e795b admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:42.419788 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22178de0-d5d3-4d5d-b1bc-d6d9ce3e795b admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:42.424986 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001ebc02-634a-417d-b144-d58ce33af125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:42.425579 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001ebc02-634a-417d-b144-d58ce33af125 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:42.425858 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001ebc02-634a-417d-b144-d58ce33af125 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:42.426131 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001ebc02-634a-417d-b144-d58ce33af125 admin admin] PATH_INFO: `/v3/projects` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:42.426750 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001ebc02-634a-417d-b144-d58ce33af125 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:42.428672 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001ebc02-634a-417d-b144-d58ce33af125 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:42.454626 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1455/2889] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 9 12:24:42 2022] POST /identity/v3/projects => generated 385 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 12:24:42.458683 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c909a08f-9a48-4f7f-8ad0-3ab7f758955a None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:42.470225 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c909a08f-9a48-4f7f-8ad0-3ab7f758955a None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:42.524632 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb27d424-c40d-4c07-ba6a-9268a2119aa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:42.525157 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb27d424-c40d-4c07-ba6a-9268a2119aa8 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:42.525432 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb27d424-c40d-4c07-ba6a-9268a2119aa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:42.525697 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb27d424-c40d-4c07-ba6a-9268a2119aa8 admin admin] PATH_INFO: `/v3/users` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:42.526318 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb27d424-c40d-4c07-ba6a-9268a2119aa8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:42.528249 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb27d424-c40d-4c07-ba6a-9268a2119aa8 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:42.550517 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1435/2890] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:24:42 2022] POST /identity/v3/users => generated 299 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:24:42.554671 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001ebc02-634a-417d-b144-d58ce33af125 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:42.565224 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001ebc02-634a-417d-b144-d58ce33af125 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:42.569924 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac116f0f-c439-47bb-bec4-72f0220bfb35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:42.570520 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac116f0f-c439-47bb-bec4-72f0220bfb35 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:42.570795 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac116f0f-c439-47bb-bec4-72f0220bfb35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:42.571153 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac116f0f-c439-47bb-bec4-72f0220bfb35 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:42.571792 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac116f0f-c439-47bb-bec4-72f0220bfb35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:42.573360 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac116f0f-c439-47bb-bec4-72f0220bfb35 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:42.583904 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1456/2891] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:24:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:24:42.587973 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb27d424-c40d-4c07-ba6a-9268a2119aa8 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:42.599033 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb27d424-c40d-4c07-ba6a-9268a2119aa8 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:42.603503 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84e02f2-24d1-4bcd-bd95-667e082b0d23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:42.604195 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84e02f2-24d1-4bcd-bd95-667e082b0d23 admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:42.604474 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84e02f2-24d1-4bcd-bd95-667e082b0d23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:42.604739 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84e02f2-24d1-4bcd-bd95-667e082b0d23 admin admin] PATH_INFO: `/v3/projects/9cd67cb0e53e40ef9b0e0d50473b3ed5/users/836f97416b7f4ac08a0bfe9b0c8dc589/roles/3f2ca33902a04b6dbc2ef7d06d7289d3` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:42.620305 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84e02f2-24d1-4bcd-bd95-667e082b0d23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9cd67cb0e53e40ef9b0e0d50473b3ed5, user_id=836f97416b7f4ac08a0bfe9b0c8dc589, role_id=3f2ca33902a04b6dbc2ef7d06d7289d3)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:42.623264 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84e02f2-24d1-4bcd-bd95-667e082b0d23 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:42.636330 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1436/2892] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:24:42 2022] PUT /identity/v3/projects/9cd67cb0e53e40ef9b0e0d50473b3ed5/users/836f97416b7f4ac08a0bfe9b0c8dc589/roles/3f2ca33902a04b6dbc2ef7d06d7289d3 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:24:42.640687 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac116f0f-c439-47bb-bec4-72f0220bfb35 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:42.651057 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac116f0f-c439-47bb-bec4-72f0220bfb35 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:42.706267 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8eaefbc-a8b7-41cd-89f4-2806760420ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:42.706863 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eaefbc-a8b7-41cd-89f4-2806760420ff admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:42.707167 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eaefbc-a8b7-41cd-89f4-2806760420ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:42.707435 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eaefbc-a8b7-41cd-89f4-2806760420ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:42.708056 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8eaefbc-a8b7-41cd-89f4-2806760420ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:42.709631 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8eaefbc-a8b7-41cd-89f4-2806760420ff admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:42.720164 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1457/2893] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:24:42 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 12:24:42.724563 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84e02f2-24d1-4bcd-bd95-667e082b0d23 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:42.735534 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84e02f2-24d1-4bcd-bd95-667e082b0d23 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:42.740080 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8d81da-7921-4cf4-b76d-c19161226dd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:42.740730 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8d81da-7921-4cf4-b76d-c19161226dd7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:42.741006 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8d81da-7921-4cf4-b76d-c19161226dd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:42.741279 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8d81da-7921-4cf4-b76d-c19161226dd7 admin admin] PATH_INFO: `/v3/projects/9cd67cb0e53e40ef9b0e0d50473b3ed5/users/836f97416b7f4ac08a0bfe9b0c8dc589/roles/3f2ca33902a04b6dbc2ef7d06d7289d3` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:42.743215 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8d81da-7921-4cf4-b76d-c19161226dd7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9cd67cb0e53e40ef9b0e0d50473b3ed5, user_id=836f97416b7f4ac08a0bfe9b0c8dc589, role_id=3f2ca33902a04b6dbc2ef7d06d7289d3)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:42.745842 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8d81da-7921-4cf4-b76d-c19161226dd7 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:42.756409 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1437/2894] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:24:42 2022] PUT /identity/v3/projects/9cd67cb0e53e40ef9b0e0d50473b3ed5/users/836f97416b7f4ac08a0bfe9b0c8dc589/roles/3f2ca33902a04b6dbc2ef7d06d7289d3 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:24:42.768593 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62cd035-4392-4c6c-898d-64648af0c92a None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:42.768884 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62cd035-4392-4c6c-898d-64648af0c92a None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:42.769166 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62cd035-4392-4c6c-898d-64648af0c92a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:42.796878 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-c62cd035-4392-4c6c-898d-64648af0c92a None None] MFA Rules not processed for user `836f97416b7f4ac08a0bfe9b0c8dc589`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:24:42.798025 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-c62cd035-4392-4c6c-898d-64648af0c92a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:24:42.837945 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:24:42.868738 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1458/2895] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:24:42 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:24:42.880524 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8d81da-7921-4cf4-b76d-c19161226dd7 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:42.882268 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-bd8d81da-7921-4cf4-b76d-c19161226dd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:24:42.897511 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:24:42.942279 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8d81da-7921-4cf4-b76d-c19161226dd7 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:42.945863 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28994518-56d1-4a09-9913-9e5c6dc58505 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535c258100df40b58a5ee44b2c21d621', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:42.946453 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28994518-56d1-4a09-9913-9e5c6dc58505 service cinder] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:42.946724 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28994518-56d1-4a09-9913-9e5c6dc58505 service cinder] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:42.947017 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28994518-56d1-4a09-9913-9e5c6dc58505 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:43.002252 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28994518-56d1-4a09-9913-9e5c6dc58505 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:43.003709 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28994518-56d1-4a09-9913-9e5c6dc58505 service cinder] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:43.012392 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1438/2896] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:24:42 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:24:44.510419 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62cd035-4392-4c6c-898d-64648af0c92a None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:44.511871 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-c62cd035-4392-4c6c-898d-64648af0c92a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:24:44.524128 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62cd035-4392-4c6c-898d-64648af0c92a None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:44.527692 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f368f2c-eda4-4022-9bd6-8fce5b36cc5a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535c258100df40b58a5ee44b2c21d621', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:44.528284 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f368f2c-eda4-4022-9bd6-8fce5b36cc5a service cinder] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:44.528641 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f368f2c-eda4-4022-9bd6-8fce5b36cc5a service cinder] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:44.529562 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f368f2c-eda4-4022-9bd6-8fce5b36cc5a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:44.585737 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f368f2c-eda4-4022-9bd6-8fce5b36cc5a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:44.587110 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f368f2c-eda4-4022-9bd6-8fce5b36cc5a service cinder] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:44.596826 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1459/2897] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:24:44 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:24:45.120335 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28994518-56d1-4a09-9913-9e5c6dc58505 service cinder] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:45.129516 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28994518-56d1-4a09-9913-9e5c6dc58505 service cinder] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:45.133076 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205c7f35-240d-4f49-aa54-4bc8d1b49cc8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535c258100df40b58a5ee44b2c21d621', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:45.133664 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205c7f35-240d-4f49-aa54-4bc8d1b49cc8 service cinder] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:45.133941 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205c7f35-240d-4f49-aa54-4bc8d1b49cc8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:45.134206 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205c7f35-240d-4f49-aa54-4bc8d1b49cc8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:45.143299 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205c7f35-240d-4f49-aa54-4bc8d1b49cc8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:45.144711 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205c7f35-240d-4f49-aa54-4bc8d1b49cc8 service cinder] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:45.154153 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1439/2898] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:24:45 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:24:49.022701 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f368f2c-eda4-4022-9bd6-8fce5b36cc5a service cinder] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:49.083453 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f368f2c-eda4-4022-9bd6-8fce5b36cc5a service cinder] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:49.087891 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcd0c0f-cba7-4509-b3c9-9e51caaed5c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:49.088503 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcd0c0f-cba7-4509-b3c9-9e51caaed5c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:49.088777 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcd0c0f-cba7-4509-b3c9-9e51caaed5c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:49.089039 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcd0c0f-cba7-4509-b3c9-9e51caaed5c9 admin admin] PATH_INFO: `/v3/users/cb766a30bf634ad9ad51253e417a77a2` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:49.090155 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcd0c0f-cba7-4509-b3c9-9e51caaed5c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb766a30bf634ad9ad51253e417a77a2)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:49.092592 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcd0c0f-cba7-4509-b3c9-9e51caaed5c9 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:49.137254 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-2fcd0c0f-cba7-4509-b3c9-9e51caaed5c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb766a30bf634ad9ad51253e417a77a2'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:49.144451 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-2fcd0c0f-cba7-4509-b3c9-9e51caaed5c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb766a30bf634ad9ad51253e417a77a2'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:49.157027 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-2fcd0c0f-cba7-4509-b3c9-9e51caaed5c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb766a30bf634ad9ad51253e417a77a2'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:49.158668 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:24:49.165118 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-2fcd0c0f-cba7-4509-b3c9-9e51caaed5c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb766a30bf634ad9ad51253e417a77a2'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:49.166077 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-2fcd0c0f-cba7-4509-b3c9-9e51caaed5c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb766a30bf634ad9ad51253e417a77a2'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:49.172571 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1460/2899] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 9 12:24:49 2022] DELETE /identity/v3/users/cb766a30bf634ad9ad51253e417a77a2 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:24:49.177451 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d14c3c-333a-40c6-9789-fb9179e8bc15 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:49.177913 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d14c3c-333a-40c6-9789-fb9179e8bc15 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:49.178014 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d14c3c-333a-40c6-9789-fb9179e8bc15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:49.196408 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-76d14c3c-333a-40c6-9789-fb9179e8bc15 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:24:49.197610 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-76d14c3c-333a-40c6-9789-fb9179e8bc15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:24:49.255669 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1440/2900] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:24:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:24:49.265471 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcd0c0f-cba7-4509-b3c9-9e51caaed5c9 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:49.266892 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-2fcd0c0f-cba7-4509-b3c9-9e51caaed5c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:24:49.324448 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcd0c0f-cba7-4509-b3c9-9e51caaed5c9 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:49.328236 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d50d7ae-2649-45c4-af49-a18a1216bae1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:49.328835 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d50d7ae-2649-45c4-af49-a18a1216bae1 service neutron] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:49.329112 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d50d7ae-2649-45c4-af49-a18a1216bae1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:49.329382 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d50d7ae-2649-45c4-af49-a18a1216bae1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:49.338178 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d50d7ae-2649-45c4-af49-a18a1216bae1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:49.339643 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d50d7ae-2649-45c4-af49-a18a1216bae1 service neutron] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:49.349408 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1461/2901] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:24:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:24:49.613289 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d14c3c-333a-40c6-9789-fb9179e8bc15 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:49.614728 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-76d14c3c-333a-40c6-9789-fb9179e8bc15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:24:49.626767 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d14c3c-333a-40c6-9789-fb9179e8bc15 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:49.631568 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c088b7c-798f-4ba8-a9f8-5d2c50240b43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:49.632231 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c088b7c-798f-4ba8-a9f8-5d2c50240b43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:49.632506 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c088b7c-798f-4ba8-a9f8-5d2c50240b43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:49.632777 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c088b7c-798f-4ba8-a9f8-5d2c50240b43 admin admin] PATH_INFO: `/v3/projects/34ba228e291d49fe8f431661207976eb` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:49.639731 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c088b7c-798f-4ba8-a9f8-5d2c50240b43 admin admin] RBAC: Authorizing `identity:delete_project(project_id=34ba228e291d49fe8f431661207976eb)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:49.641757 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c088b7c-798f-4ba8-a9f8-5d2c50240b43 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:49.692202 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-5c088b7c-798f-4ba8-a9f8-5d2c50240b43 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34ba228e291d49fe8f431661207976eb', 'request_id': 'req-5c088b7c-798f-4ba8-a9f8-5d2c50240b43'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:49.693347 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:24:49.699163 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-5c088b7c-798f-4ba8-a9f8-5d2c50240b43 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34ba228e291d49fe8f431661207976eb', 'request_id': 'req-5c088b7c-798f-4ba8-a9f8-5d2c50240b43'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:49.708344 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-5c088b7c-798f-4ba8-a9f8-5d2c50240b43 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34ba228e291d49fe8f431661207976eb', 'request_id': 'req-5c088b7c-798f-4ba8-a9f8-5d2c50240b43'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:49.719893 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-5c088b7c-798f-4ba8-a9f8-5d2c50240b43 admin admin] The token cache is being invalidate because project 34ba228e291d49fe8f431661207976eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:24:49.720180 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-5c088b7c-798f-4ba8-a9f8-5d2c50240b43 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:49.720855 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-5c088b7c-798f-4ba8-a9f8-5d2c50240b43 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:49.722342 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1441/2902] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 9 12:24:49 2022] DELETE /identity/v3/projects/34ba228e291d49fe8f431661207976eb => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:24:49.726372 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d50d7ae-2649-45c4-af49-a18a1216bae1 service neutron] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:49.736471 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d50d7ae-2649-45c4-af49-a18a1216bae1 service neutron] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:49.792916 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0207701d-032e-4dfd-a818-ef43ceba14f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:49.793463 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0207701d-032e-4dfd-a818-ef43ceba14f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:49.793740 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0207701d-032e-4dfd-a818-ef43ceba14f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:49.794007 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0207701d-032e-4dfd-a818-ef43ceba14f1 admin admin] PATH_INFO: `/v3/users/75fde45dcf054c808a484902db439191` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:49.795808 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0207701d-032e-4dfd-a818-ef43ceba14f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=75fde45dcf054c808a484902db439191)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:49.797711 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0207701d-032e-4dfd-a818-ef43ceba14f1 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:49.842726 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-0207701d-032e-4dfd-a818-ef43ceba14f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75fde45dcf054c808a484902db439191'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:49.849598 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-0207701d-032e-4dfd-a818-ef43ceba14f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75fde45dcf054c808a484902db439191'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:49.861908 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-0207701d-032e-4dfd-a818-ef43ceba14f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75fde45dcf054c808a484902db439191'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:49.869365 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-0207701d-032e-4dfd-a818-ef43ceba14f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75fde45dcf054c808a484902db439191'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:49.869981 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-0207701d-032e-4dfd-a818-ef43ceba14f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75fde45dcf054c808a484902db439191'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:49.875902 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1462/2903] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 9 12:24:49 2022] DELETE /identity/v3/users/75fde45dcf054c808a484902db439191 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:24:50.156960 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c088b7c-798f-4ba8-a9f8-5d2c50240b43 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:50.158400 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-5c088b7c-798f-4ba8-a9f8-5d2c50240b43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:24:50.173195 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c088b7c-798f-4ba8-a9f8-5d2c50240b43 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:50.229939 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ea2679-e545-42e2-bf6f-161ac511a0d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:50.230591 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ea2679-e545-42e2-bf6f-161ac511a0d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:50.230840 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ea2679-e545-42e2-bf6f-161ac511a0d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:50.231143 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ea2679-e545-42e2-bf6f-161ac511a0d1 admin admin] PATH_INFO: `/v3/projects/6d428fb6c1f84ba8b0974c1b1609b62a` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:50.233142 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ea2679-e545-42e2-bf6f-161ac511a0d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d428fb6c1f84ba8b0974c1b1609b62a)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:50.235102 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ea2679-e545-42e2-bf6f-161ac511a0d1 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:50.280714 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-37ea2679-e545-42e2-bf6f-161ac511a0d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d428fb6c1f84ba8b0974c1b1609b62a', 'request_id': 'req-37ea2679-e545-42e2-bf6f-161ac511a0d1'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:50.286153 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-37ea2679-e545-42e2-bf6f-161ac511a0d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d428fb6c1f84ba8b0974c1b1609b62a', 'request_id': 'req-37ea2679-e545-42e2-bf6f-161ac511a0d1'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:50.293319 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-37ea2679-e545-42e2-bf6f-161ac511a0d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d428fb6c1f84ba8b0974c1b1609b62a', 'request_id': 'req-37ea2679-e545-42e2-bf6f-161ac511a0d1'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:50.304752 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-37ea2679-e545-42e2-bf6f-161ac511a0d1 admin admin] The token cache is being invalidate because project 6d428fb6c1f84ba8b0974c1b1609b62a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:24:50.305039 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-37ea2679-e545-42e2-bf6f-161ac511a0d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:50.305674 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-37ea2679-e545-42e2-bf6f-161ac511a0d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:50.307188 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1442/2904] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 9 12:24:50 2022] DELETE /identity/v3/projects/6d428fb6c1f84ba8b0974c1b1609b62a => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:24:50.311470 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0207701d-032e-4dfd-a818-ef43ceba14f1 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:50.312932 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-0207701d-032e-4dfd-a818-ef43ceba14f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:24:50.325284 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0207701d-032e-4dfd-a818-ef43ceba14f1 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:50.334696 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:24:50.383194 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ae190a-f51d-4ede-95c7-040d2eebcfe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:50.383815 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ae190a-f51d-4ede-95c7-040d2eebcfe5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:50.384089 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ae190a-f51d-4ede-95c7-040d2eebcfe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:50.384358 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ae190a-f51d-4ede-95c7-040d2eebcfe5 admin admin] PATH_INFO: `/v3/users/836f97416b7f4ac08a0bfe9b0c8dc589` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:50.385453 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ae190a-f51d-4ede-95c7-040d2eebcfe5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=836f97416b7f4ac08a0bfe9b0c8dc589)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:50.387389 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ae190a-f51d-4ede-95c7-040d2eebcfe5 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:50.434123 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-19ae190a-f51d-4ede-95c7-040d2eebcfe5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '836f97416b7f4ac08a0bfe9b0c8dc589'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:50.441909 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-19ae190a-f51d-4ede-95c7-040d2eebcfe5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '836f97416b7f4ac08a0bfe9b0c8dc589'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:50.454171 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-19ae190a-f51d-4ede-95c7-040d2eebcfe5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '836f97416b7f4ac08a0bfe9b0c8dc589'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:50.461537 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-19ae190a-f51d-4ede-95c7-040d2eebcfe5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '836f97416b7f4ac08a0bfe9b0c8dc589'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:50.462171 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-19ae190a-f51d-4ede-95c7-040d2eebcfe5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '836f97416b7f4ac08a0bfe9b0c8dc589'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:50.468597 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1463/2905] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 9 12:24:50 2022] DELETE /identity/v3/users/836f97416b7f4ac08a0bfe9b0c8dc589 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:24:50.738865 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ea2679-e545-42e2-bf6f-161ac511a0d1 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:50.740326 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-37ea2679-e545-42e2-bf6f-161ac511a0d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:24:50.753622 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ea2679-e545-42e2-bf6f-161ac511a0d1 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:50.811164 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c54590e-f2ca-4635-b1b1-9d90c544590e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:50.811798 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c54590e-f2ca-4635-b1b1-9d90c544590e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:50.812071 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c54590e-f2ca-4635-b1b1-9d90c544590e admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:50.812338 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c54590e-f2ca-4635-b1b1-9d90c544590e admin admin] PATH_INFO: `/v3/projects/9cd67cb0e53e40ef9b0e0d50473b3ed5` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:50.813453 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c54590e-f2ca-4635-b1b1-9d90c544590e admin admin] RBAC: Authorizing `identity:delete_project(project_id=9cd67cb0e53e40ef9b0e0d50473b3ed5)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:50.815379 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c54590e-f2ca-4635-b1b1-9d90c544590e admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:50.825341 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:24:50.861279 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-3c54590e-f2ca-4635-b1b1-9d90c544590e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9cd67cb0e53e40ef9b0e0d50473b3ed5', 'request_id': 'req-3c54590e-f2ca-4635-b1b1-9d90c544590e'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:50.866590 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-3c54590e-f2ca-4635-b1b1-9d90c544590e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9cd67cb0e53e40ef9b0e0d50473b3ed5', 'request_id': 'req-3c54590e-f2ca-4635-b1b1-9d90c544590e'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:50.874092 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-3c54590e-f2ca-4635-b1b1-9d90c544590e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9cd67cb0e53e40ef9b0e0d50473b3ed5', 'request_id': 'req-3c54590e-f2ca-4635-b1b1-9d90c544590e'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:50.884798 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-3c54590e-f2ca-4635-b1b1-9d90c544590e admin admin] The token cache is being invalidate because project 9cd67cb0e53e40ef9b0e0d50473b3ed5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:24:50.885085 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-3c54590e-f2ca-4635-b1b1-9d90c544590e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:50.885727 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-3c54590e-f2ca-4635-b1b1-9d90c544590e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:24:50.887258 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1443/2906] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 9 12:24:50 2022] DELETE /identity/v3/projects/9cd67cb0e53e40ef9b0e0d50473b3ed5 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:24:50.892835 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368cfd6c-e3bb-48f1-9cb5-60c96db43fbb None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:50.893122 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368cfd6c-e3bb-48f1-9cb5-60c96db43fbb None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:50.893393 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368cfd6c-e3bb-48f1-9cb5-60c96db43fbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:50.912826 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-368cfd6c-e3bb-48f1-9cb5-60c96db43fbb None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:24:50.914036 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-368cfd6c-e3bb-48f1-9cb5-60c96db43fbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:24:50.972353 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1464/2907] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:24:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:24:50.980102 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8965000-d36e-4c20-b674-6747f98be489 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:50.980390 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8965000-d36e-4c20-b674-6747f98be489 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:50.980661 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8965000-d36e-4c20-b674-6747f98be489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:51.001141 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-a8965000-d36e-4c20-b674-6747f98be489 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:24:51.002350 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-a8965000-d36e-4c20-b674-6747f98be489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:24:51.010071 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1444/2908] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:24:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:24:51.014422 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368cfd6c-e3bb-48f1-9cb5-60c96db43fbb None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:51.024055 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368cfd6c-e3bb-48f1-9cb5-60c96db43fbb None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:51.028610 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a66db9-45c5-4e5f-b92c-073d9a879bc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:51.029459 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a66db9-45c5-4e5f-b92c-073d9a879bc0 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:51.029756 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a66db9-45c5-4e5f-b92c-073d9a879bc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:51.030014 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a66db9-45c5-4e5f-b92c-073d9a879bc0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:51.030659 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a66db9-45c5-4e5f-b92c-073d9a879bc0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81591) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 12:24:51.030996 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a66db9-45c5-4e5f-b92c-073d9a879bc0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:51.032590 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a66db9-45c5-4e5f-b92c-073d9a879bc0 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:51.044005 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1465/2909] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 9 12:24:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 12:24:51.048967 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8965000-d36e-4c20-b674-6747f98be489 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:51.058308 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8965000-d36e-4c20-b674-6747f98be489 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:51.063295 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c64b69-0ebd-4f70-93ba-3c043a085d29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:51.063887 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c64b69-0ebd-4f70-93ba-3c043a085d29 admin admin] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:51.064164 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c64b69-0ebd-4f70-93ba-3c043a085d29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:51.064436 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c64b69-0ebd-4f70-93ba-3c043a085d29 admin admin] PATH_INFO: `/v3/projects` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:51.065061 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c64b69-0ebd-4f70-93ba-3c043a085d29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:51.066990 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c64b69-0ebd-4f70-93ba-3c043a085d29 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:51.094282 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1445/2910] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 9 12:24:51 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:24:51.098359 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a66db9-45c5-4e5f-b92c-073d9a879bc0 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:51.108356 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a66db9-45c5-4e5f-b92c-073d9a879bc0 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:51.125427 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:24:51.166697 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a54a10-f209-470a-9484-3acd36b9d814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:51.167326 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a54a10-f209-470a-9484-3acd36b9d814 admin admin] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:51.167597 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a54a10-f209-470a-9484-3acd36b9d814 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:51.167863 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a54a10-f209-470a-9484-3acd36b9d814 admin admin] PATH_INFO: `/v3/users` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:51.168513 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a54a10-f209-470a-9484-3acd36b9d814 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:51.170467 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a54a10-f209-470a-9484-3acd36b9d814 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:51.191233 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1466/2911] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 9 12:24:51 2022] POST /identity/v3/users => generated 292 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 12:24:51.195350 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c64b69-0ebd-4f70-93ba-3c043a085d29 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:51.206342 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c64b69-0ebd-4f70-93ba-3c043a085d29 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:51.210887 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cfd0ff-2aa5-439b-95fc-e81e075f48ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:51.211508 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cfd0ff-2aa5-439b-95fc-e81e075f48ba admin admin] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:51.211785 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cfd0ff-2aa5-439b-95fc-e81e075f48ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:51.212054 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cfd0ff-2aa5-439b-95fc-e81e075f48ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:51.212688 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cfd0ff-2aa5-439b-95fc-e81e075f48ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:51.214282 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cfd0ff-2aa5-439b-95fc-e81e075f48ba admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:51.224769 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1446/2912] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:24:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:24:51.228725 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a54a10-f209-470a-9484-3acd36b9d814 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:51.238923 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a54a10-f209-470a-9484-3acd36b9d814 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:51.243556 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9000e65-5e9c-4175-acaf-ab2efe5bc372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:51.244144 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9000e65-5e9c-4175-acaf-ab2efe5bc372 admin admin] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:51.244425 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9000e65-5e9c-4175-acaf-ab2efe5bc372 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:51.244695 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9000e65-5e9c-4175-acaf-ab2efe5bc372 admin admin] PATH_INFO: `/v3/roles` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:51.245329 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9000e65-5e9c-4175-acaf-ab2efe5bc372 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:51.246937 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9000e65-5e9c-4175-acaf-ab2efe5bc372 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:51.257718 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1467/2913] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 9 12:24:51 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 12:24:51.261783 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cfd0ff-2aa5-439b-95fc-e81e075f48ba admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:51.272741 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cfd0ff-2aa5-439b-95fc-e81e075f48ba admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:51.277397 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466e14c8-0d16-4c49-af4e-117eaba26930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:51.278046 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466e14c8-0d16-4c49-af4e-117eaba26930 admin admin] REQUEST_METHOD: `PUT` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:51.278324 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466e14c8-0d16-4c49-af4e-117eaba26930 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:51.279197 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466e14c8-0d16-4c49-af4e-117eaba26930 admin admin] PATH_INFO: `/v3/projects/9b0b40d750ab4721a1339c8d2c8d5052/users/52fe2bbd491f40b2a9e99ef2a33c9e84/roles/eecd2f160dd844f5906f3d345780c501` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:51.293510 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466e14c8-0d16-4c49-af4e-117eaba26930 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b0b40d750ab4721a1339c8d2c8d5052, user_id=52fe2bbd491f40b2a9e99ef2a33c9e84, role_id=eecd2f160dd844f5906f3d345780c501)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:51.296308 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466e14c8-0d16-4c49-af4e-117eaba26930 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:51.309307 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1447/2914] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 9 12:24:51 2022] PUT /identity/v3/projects/9b0b40d750ab4721a1339c8d2c8d5052/users/52fe2bbd491f40b2a9e99ef2a33c9e84/roles/eecd2f160dd844f5906f3d345780c501 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:24:51.314561 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3bdbec-8f24-4468-a175-d247a3150111 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:51.315079 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3bdbec-8f24-4468-a175-d247a3150111 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:51.315185 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3bdbec-8f24-4468-a175-d247a3150111 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:51.334205 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-ca3bdbec-8f24-4468-a175-d247a3150111 None None] MFA Rules not processed for user `136011715ec34418adc0f536f6c7357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:24:51.335413 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-ca3bdbec-8f24-4468-a175-d247a3150111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:24:51.393021 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1468/2915] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:24:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:24:51.402599 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466e14c8-0d16-4c49-af4e-117eaba26930 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:51.404431 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-466e14c8-0d16-4c49-af4e-117eaba26930 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:24:51.462157 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466e14c8-0d16-4c49-af4e-117eaba26930 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:51.465799 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60e5fd2-d7f2-4bf7-8432-91f30f1087d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:51.466484 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60e5fd2-d7f2-4bf7-8432-91f30f1087d7 service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:51.466853 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60e5fd2-d7f2-4bf7-8432-91f30f1087d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:51.467096 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60e5fd2-d7f2-4bf7-8432-91f30f1087d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:51.475894 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60e5fd2-d7f2-4bf7-8432-91f30f1087d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:51.477324 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60e5fd2-d7f2-4bf7-8432-91f30f1087d7 service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:51.486914 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1448/2916] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:24:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:24:52.116083 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabf7332-484c-47e9-911a-2ad5ca26773a None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:52.116366 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabf7332-484c-47e9-911a-2ad5ca26773a None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:52.116635 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabf7332-484c-47e9-911a-2ad5ca26773a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:52.145902 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-eabf7332-484c-47e9-911a-2ad5ca26773a None None] MFA Rules not processed for user `52fe2bbd491f40b2a9e99ef2a33c9e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:24:52.147174 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-eabf7332-484c-47e9-911a-2ad5ca26773a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:24:52.215844 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1469/2917] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:24:52 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:24:52.227493 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60e5fd2-d7f2-4bf7-8432-91f30f1087d7 service neutron] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:52.229309 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-e60e5fd2-d7f2-4bf7-8432-91f30f1087d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:24:52.271875 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:24:52.289976 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60e5fd2-d7f2-4bf7-8432-91f30f1087d7 service neutron] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:52.293689 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd9a1c3-6f1e-40a4-90d8-7c796f88ca5b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '535c258100df40b58a5ee44b2c21d621', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:52.294882 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd9a1c3-6f1e-40a4-90d8-7c796f88ca5b service cinder] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:52.295199 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd9a1c3-6f1e-40a4-90d8-7c796f88ca5b service cinder] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:52.295471 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd9a1c3-6f1e-40a4-90d8-7c796f88ca5b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:52.304271 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd9a1c3-6f1e-40a4-90d8-7c796f88ca5b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:52.305699 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd9a1c3-6f1e-40a4-90d8-7c796f88ca5b service cinder] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:52.314958 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1449/2918] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:24:52 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:24:53.672773 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb940e94-025b-4177-b35b-16dc81429f63 None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:53.673319 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb940e94-025b-4177-b35b-16dc81429f63 None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:53.673319 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb940e94-025b-4177-b35b-16dc81429f63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:53.692265 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-eb940e94-025b-4177-b35b-16dc81429f63 None None] MFA Rules not processed for user `52fe2bbd491f40b2a9e99ef2a33c9e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:24:53.693461 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-eb940e94-025b-4177-b35b-16dc81429f63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:24:53.700204 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1470/2919] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:24:53 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:24:53.935399 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd9a1c3-6f1e-40a4-90d8-7c796f88ca5b service cinder] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:24:53.936802 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-afd9a1c3-6f1e-40a4-90d8-7c796f88ca5b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:24:53.948607 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd9a1c3-6f1e-40a4-90d8-7c796f88ca5b service cinder] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:24:53.952157 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ac1fed-9cd8-48f7-a61c-f945f7cf7e55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:24:53.952739 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ac1fed-9cd8-48f7-a61c-f945f7cf7e55 service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:24:53.953015 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ac1fed-9cd8-48f7-a61c-f945f7cf7e55 service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:24:53.953285 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ac1fed-9cd8-48f7-a61c-f945f7cf7e55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:24:53.954961 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-c3ac1fed-9cd8-48f7-a61c-f945f7cf7e55 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:24:53.964183 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ac1fed-9cd8-48f7-a61c-f945f7cf7e55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:24:53.965585 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ac1fed-9cd8-48f7-a61c-f945f7cf7e55 service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:24:53.974068 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1450/2920] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:24:53 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:25:03.686839 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=87,keystone:INSERT=3 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:25:03.970496 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=15 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:25:26.210713 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a095995-243d-429a-8480-1724707cd98a None None] REQUEST_METHOD: `POST` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:25:26.211322 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a095995-243d-429a-8480-1724707cd98a None None] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:25:26.211322 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a095995-243d-429a-8480-1724707cd98a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:25:26.229782 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.auth.core [None req-0a095995-243d-429a-8480-1724707cd98a None None] MFA Rules not processed for user `e2391d64c9304317bfb53e79ee0ca764`. Rule list: `[]` (Enabled: `True`). {{(pid=81591) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:25:26.231318 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-0a095995-243d-429a-8480-1724707cd98a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:25:26.287227 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1471/2921] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:25:26 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:25:36.280598 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=25 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:25:57.274572 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe38a3c7-f44c-4d24-99b8-28245e88f6f8 None None] REQUEST_METHOD: `POST` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:25:57.275114 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe38a3c7-f44c-4d24-99b8-28245e88f6f8 None None] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:25:57.275242 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe38a3c7-f44c-4d24-99b8-28245e88f6f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:25:57.294601 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.auth.core [None req-fe38a3c7-f44c-4d24-99b8-28245e88f6f8 None None] MFA Rules not processed for user `52fe2bbd491f40b2a9e99ef2a33c9e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81590) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 12:25:57.295846 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-fe38a3c7-f44c-4d24-99b8-28245e88f6f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:25:57.302866 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1451/2922] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 9 12:25:57 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 12:26:07.288993 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=3 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:26:30.681680 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a095995-243d-429a-8480-1724707cd98a None None] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:26:30.683142 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-0a095995-243d-429a-8480-1724707cd98a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:26:30.754285 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a095995-243d-429a-8480-1724707cd98a None None] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:26:30.760096 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5577a55d-05ea-4d6f-b51e-5ca5eb771c84 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3adb2f0583c4ff782049dacdb3ee556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:26:30.760711 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5577a55d-05ea-4d6f-b51e-5ca5eb771c84 service placement] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:26:30.760997 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5577a55d-05ea-4d6f-b51e-5ca5eb771c84 service placement] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:26:30.761255 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5577a55d-05ea-4d6f-b51e-5ca5eb771c84 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:26:30.763029 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-5577a55d-05ea-4d6f-b51e-5ca5eb771c84 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:26:30.776361 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5577a55d-05ea-4d6f-b51e-5ca5eb771c84 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:26:30.778448 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5577a55d-05ea-4d6f-b51e-5ca5eb771c84 service placement] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:26:30.789277 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1472/2923] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:26:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 12:26:40.784305 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=32 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:26:55.580511 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe38a3c7-f44c-4d24-99b8-28245e88f6f8 None None] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:26:55.581969 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-fe38a3c7-f44c-4d24-99b8-28245e88f6f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:26:55.596601 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe38a3c7-f44c-4d24-99b8-28245e88f6f8 None None] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:26:55.602234 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b0684a-c205-4f6c-b708-10152d5a13da service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3adb2f0583c4ff782049dacdb3ee556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:26:55.602852 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b0684a-c205-4f6c-b708-10152d5a13da service placement] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:26:55.603164 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b0684a-c205-4f6c-b708-10152d5a13da service placement] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:26:55.603464 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b0684a-c205-4f6c-b708-10152d5a13da service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:26:55.610634 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-e5b0684a-c205-4f6c-b708-10152d5a13da service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:26:55.623386 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b0684a-c205-4f6c-b708-10152d5a13da service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:26:55.629257 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b0684a-c205-4f6c-b708-10152d5a13da service placement] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:26:55.636693 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1452/2924] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:26:55 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:26:55.647854 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5577a55d-05ea-4d6f-b51e-5ca5eb771c84 service placement] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:26:55.660010 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5577a55d-05ea-4d6f-b51e-5ca5eb771c84 service placement] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:26:55.664604 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4593d0ad-6759-44a5-9e4c-096c86ddd82e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:26:55.665235 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4593d0ad-6759-44a5-9e4c-096c86ddd82e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:26:55.665509 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4593d0ad-6759-44a5-9e4c-096c86ddd82e admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:26:55.665777 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4593d0ad-6759-44a5-9e4c-096c86ddd82e admin admin] PATH_INFO: `/v3/users/52fe2bbd491f40b2a9e99ef2a33c9e84` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:26:55.667131 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4593d0ad-6759-44a5-9e4c-096c86ddd82e admin admin] RBAC: Authorizing `identity:delete_user(user_id=52fe2bbd491f40b2a9e99ef2a33c9e84)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:26:55.669535 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4593d0ad-6759-44a5-9e4c-096c86ddd82e admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:26:55.716613 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-4593d0ad-6759-44a5-9e4c-096c86ddd82e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52fe2bbd491f40b2a9e99ef2a33c9e84'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:26:55.723614 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-4593d0ad-6759-44a5-9e4c-096c86ddd82e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52fe2bbd491f40b2a9e99ef2a33c9e84'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:26:55.735236 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-4593d0ad-6759-44a5-9e4c-096c86ddd82e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52fe2bbd491f40b2a9e99ef2a33c9e84'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:26:55.742773 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-4593d0ad-6759-44a5-9e4c-096c86ddd82e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52fe2bbd491f40b2a9e99ef2a33c9e84'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:26:55.743413 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-4593d0ad-6759-44a5-9e4c-096c86ddd82e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52fe2bbd491f40b2a9e99ef2a33c9e84'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:26:55.749958 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1473/2925] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 9 12:26:55 2022] DELETE /identity/v3/users/52fe2bbd491f40b2a9e99ef2a33c9e84 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:26:55.924650 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b0684a-c205-4f6c-b708-10152d5a13da service placement] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:26:55.936087 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b0684a-c205-4f6c-b708-10152d5a13da service placement] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:26:55.994153 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8edcd01-4e9d-4873-92aa-f9d243a95a59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:26:55.994815 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8edcd01-4e9d-4873-92aa-f9d243a95a59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:26:55.995124 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8edcd01-4e9d-4873-92aa-f9d243a95a59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:26:55.995397 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8edcd01-4e9d-4873-92aa-f9d243a95a59 admin admin] PATH_INFO: `/v3/projects/9b0b40d750ab4721a1339c8d2c8d5052` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:26:55.996589 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8edcd01-4e9d-4873-92aa-f9d243a95a59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b0b40d750ab4721a1339c8d2c8d5052)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:26:55.998524 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8edcd01-4e9d-4873-92aa-f9d243a95a59 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:26:56.044160 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-c8edcd01-4e9d-4873-92aa-f9d243a95a59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b0b40d750ab4721a1339c8d2c8d5052', 'request_id': 'req-c8edcd01-4e9d-4873-92aa-f9d243a95a59'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:26:56.049561 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-c8edcd01-4e9d-4873-92aa-f9d243a95a59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b0b40d750ab4721a1339c8d2c8d5052', 'request_id': 'req-c8edcd01-4e9d-4873-92aa-f9d243a95a59'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:26:56.057038 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-c8edcd01-4e9d-4873-92aa-f9d243a95a59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b0b40d750ab4721a1339c8d2c8d5052', 'request_id': 'req-c8edcd01-4e9d-4873-92aa-f9d243a95a59'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:26:56.069234 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-c8edcd01-4e9d-4873-92aa-f9d243a95a59 admin admin] The token cache is being invalidate because project 9b0b40d750ab4721a1339c8d2c8d5052 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:26:56.069524 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-c8edcd01-4e9d-4873-92aa-f9d243a95a59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:26:56.070149 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-c8edcd01-4e9d-4873-92aa-f9d243a95a59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:26:56.071675 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1453/2926] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 9 12:26:55 2022] DELETE /identity/v3/projects/9b0b40d750ab4721a1339c8d2c8d5052 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:27:05.745853 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=29,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:27:06.066258 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=57,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:27:30.770748 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4593d0ad-6759-44a5-9e4c-096c86ddd82e admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:27:30.772206 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-4593d0ad-6759-44a5-9e4c-096c86ddd82e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:27:30.830465 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4593d0ad-6759-44a5-9e4c-096c86ddd82e admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:27:30.878552 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b53384-d4f0-4cf9-a85c-b0f307d049a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:27:30.879169 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b53384-d4f0-4cf9-a85c-b0f307d049a1 service neutron] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:27:30.879448 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b53384-d4f0-4cf9-a85c-b0f307d049a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:27:30.879719 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b53384-d4f0-4cf9-a85c-b0f307d049a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:27:30.881446 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-b1b53384-d4f0-4cf9-a85c-b0f307d049a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:27:30.939745 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b53384-d4f0-4cf9-a85c-b0f307d049a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:27:30.941108 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b53384-d4f0-4cf9-a85c-b0f307d049a1 service neutron] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:27:30.949927 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1474/2927] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:27:30 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:27:40.945885 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=68 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:27:44.554643 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8edcd01-4e9d-4873-92aa-f9d243a95a59 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:27:44.556427 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-c8edcd01-4e9d-4873-92aa-f9d243a95a59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:27:44.568799 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8edcd01-4e9d-4873-92aa-f9d243a95a59 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:27:44.572499 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24e4f70-ca61-41a6-adcf-67be766aaa29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '992de6f86b42495dafe73cb2c393fb4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:27:44.573095 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24e4f70-ca61-41a6-adcf-67be766aaa29 service neutron] REQUEST_METHOD: `GET` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:27:44.573399 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24e4f70-ca61-41a6-adcf-67be766aaa29 service neutron] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:27:44.573667 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24e4f70-ca61-41a6-adcf-67be766aaa29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:27:44.575437 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-e24e4f70-ca61-41a6-adcf-67be766aaa29 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:27:44.640883 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24e4f70-ca61-41a6-adcf-67be766aaa29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:27:44.642244 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24e4f70-ca61-41a6-adcf-67be766aaa29 service neutron] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:27:44.652234 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1454/2928] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:27:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:27:50.660733 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b53384-d4f0-4cf9-a85c-b0f307d049a1 service neutron] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:27:50.662290 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-b1b53384-d4f0-4cf9-a85c-b0f307d049a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:27:50.677959 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b53384-d4f0-4cf9-a85c-b0f307d049a1 service neutron] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:27:50.682868 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a255556e-2151-4638-a065-b61c7bb52eb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:27:50.683525 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a255556e-2151-4638-a065-b61c7bb52eb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:27:50.683806 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a255556e-2151-4638-a065-b61c7bb52eb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:27:50.684082 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a255556e-2151-4638-a065-b61c7bb52eb0 admin admin] PATH_INFO: `/v3/users/e2391d64c9304317bfb53e79ee0ca764` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:27:50.685201 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a255556e-2151-4638-a065-b61c7bb52eb0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2391d64c9304317bfb53e79ee0ca764)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:27:50.687203 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a255556e-2151-4638-a065-b61c7bb52eb0 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:27:50.733628 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-a255556e-2151-4638-a065-b61c7bb52eb0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2391d64c9304317bfb53e79ee0ca764'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:27:50.740636 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-a255556e-2151-4638-a065-b61c7bb52eb0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2391d64c9304317bfb53e79ee0ca764'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:27:50.752368 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-a255556e-2151-4638-a065-b61c7bb52eb0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2391d64c9304317bfb53e79ee0ca764'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:27:50.759711 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-a255556e-2151-4638-a065-b61c7bb52eb0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2391d64c9304317bfb53e79ee0ca764'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:27:50.760344 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-a255556e-2151-4638-a065-b61c7bb52eb0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2391d64c9304317bfb53e79ee0ca764'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:27:50.766870 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1475/2929] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 9 12:27:50 2022] DELETE /identity/v3/users/e2391d64c9304317bfb53e79ee0ca764 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 12:27:50.949003 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24e4f70-ca61-41a6-adcf-67be766aaa29 service neutron] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:27:50.960825 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24e4f70-ca61-41a6-adcf-67be766aaa29 service neutron] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:27:51.020049 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ed1f51-a17e-42f5-8b57-319e52acca02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:27:51.020680 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ed1f51-a17e-42f5-8b57-319e52acca02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:27:51.020956 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ed1f51-a17e-42f5-8b57-319e52acca02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:27:51.021227 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ed1f51-a17e-42f5-8b57-319e52acca02 admin admin] PATH_INFO: `/v3/projects/c9c41d17597b4d3a8ffafa3d5bb87916` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:27:51.022340 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ed1f51-a17e-42f5-8b57-319e52acca02 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9c41d17597b4d3a8ffafa3d5bb87916)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:27:51.024361 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ed1f51-a17e-42f5-8b57-319e52acca02 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:27:51.069504 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-54ed1f51-a17e-42f5-8b57-319e52acca02 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9c41d17597b4d3a8ffafa3d5bb87916', 'request_id': 'req-54ed1f51-a17e-42f5-8b57-319e52acca02'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:27:51.074979 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-54ed1f51-a17e-42f5-8b57-319e52acca02 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9c41d17597b4d3a8ffafa3d5bb87916', 'request_id': 'req-54ed1f51-a17e-42f5-8b57-319e52acca02'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:27:51.082829 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-54ed1f51-a17e-42f5-8b57-319e52acca02 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9c41d17597b4d3a8ffafa3d5bb87916', 'request_id': 'req-54ed1f51-a17e-42f5-8b57-319e52acca02'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:27:51.093644 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-54ed1f51-a17e-42f5-8b57-319e52acca02 admin admin] The token cache is being invalidate because project c9c41d17597b4d3a8ffafa3d5bb87916 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:27:51.093925 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-54ed1f51-a17e-42f5-8b57-319e52acca02 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:27:51.094544 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-54ed1f51-a17e-42f5-8b57-319e52acca02 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:27:51.096026 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1455/2930] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 9 12:27:50 2022] DELETE /identity/v3/projects/c9c41d17597b4d3a8ffafa3d5bb87916 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:27:51.099984 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a255556e-2151-4638-a065-b61c7bb52eb0 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:27:51.101437 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-a255556e-2151-4638-a065-b61c7bb52eb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:27:51.114348 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a255556e-2151-4638-a065-b61c7bb52eb0 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:27:51.170869 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533b71e8-7680-4465-ade6-7705c65c5b63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:27:51.171644 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533b71e8-7680-4465-ade6-7705c65c5b63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:27:51.171921 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533b71e8-7680-4465-ade6-7705c65c5b63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:27:51.172193 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533b71e8-7680-4465-ade6-7705c65c5b63 admin admin] PATH_INFO: `/v3/users/e7d1f7afdd42442484c3e8d93b12878c` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:27:51.173325 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533b71e8-7680-4465-ade6-7705c65c5b63 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7d1f7afdd42442484c3e8d93b12878c)` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:27:51.175307 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533b71e8-7680-4465-ade6-7705c65c5b63 admin admin] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:27:51.220775 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-533b71e8-7680-4465-ade6-7705c65c5b63 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7d1f7afdd42442484c3e8d93b12878c'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:27:51.227635 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-533b71e8-7680-4465-ade6-7705c65c5b63 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7d1f7afdd42442484c3e8d93b12878c'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:27:51.239396 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-533b71e8-7680-4465-ade6-7705c65c5b63 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7d1f7afdd42442484c3e8d93b12878c'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:27:51.241379 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:27:51.248169 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-533b71e8-7680-4465-ade6-7705c65c5b63 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7d1f7afdd42442484c3e8d93b12878c'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:27:51.249394 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.notifications [None req-533b71e8-7680-4465-ade6-7705c65c5b63 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7d1f7afdd42442484c3e8d93b12878c'} {{(pid=81591) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:27:51.256211 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1476/2931] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 9 12:27:51 2022] DELETE /identity/v3/users/e7d1f7afdd42442484c3e8d93b12878c => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:27:51.428268 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ed1f51-a17e-42f5-8b57-319e52acca02 admin admin] Authenticating user token {{(pid=81590) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:27:51.429689 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.fernet_utils [None req-54ed1f51-a17e-42f5-8b57-319e52acca02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81590) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:27:51.444200 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ed1f51-a17e-42f5-8b57-319e52acca02 admin admin] Validating token access rules against request {{(pid=81590) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:27:51.464817 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:27:51.504123 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7d9489-37f2-400e-8355-94b8bfa025e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '136011715ec34418adc0f536f6c7357c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae50911c7cf74f75b5b41fa5275c8468', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81590) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:27:51.504786 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7d9489-37f2-400e-8355-94b8bfa025e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:27:51.505066 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7d9489-37f2-400e-8355-94b8bfa025e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:27:51.505359 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7d9489-37f2-400e-8355-94b8bfa025e6 admin admin] PATH_INFO: `/v3/projects/99c91073ba464c738a6055c59d2d9ea6` {{(pid=81590) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:27:51.506509 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7d9489-37f2-400e-8355-94b8bfa025e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=99c91073ba464c738a6055c59d2d9ea6)` {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:27:51.508490 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7d9489-37f2-400e-8355-94b8bfa025e6 admin admin] RBAC: Authorization granted {{(pid=81590) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:27:51.555121 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-ae7d9489-37f2-400e-8355-94b8bfa025e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99c91073ba464c738a6055c59d2d9ea6', 'request_id': 'req-ae7d9489-37f2-400e-8355-94b8bfa025e6'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:27:51.560777 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-ae7d9489-37f2-400e-8355-94b8bfa025e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99c91073ba464c738a6055c59d2d9ea6', 'request_id': 'req-ae7d9489-37f2-400e-8355-94b8bfa025e6'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:27:51.570148 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-ae7d9489-37f2-400e-8355-94b8bfa025e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99c91073ba464c738a6055c59d2d9ea6', 'request_id': 'req-ae7d9489-37f2-400e-8355-94b8bfa025e6'} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:27:51.581626 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-ae7d9489-37f2-400e-8355-94b8bfa025e6 admin admin] The token cache is being invalidate because project 99c91073ba464c738a6055c59d2d9ea6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81590) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 12:27:51.581911 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-ae7d9489-37f2-400e-8355-94b8bfa025e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:27:51.582542 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG keystone.notifications [None req-ae7d9489-37f2-400e-8355-94b8bfa025e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81590) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 12:27:51.584881 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: [pid: 81590|app: 0|req: 1456/2932] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 9 12:27:51 2022] DELETE /identity/v3/projects/99c91073ba464c738a6055c59d2d9ea6 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 12:28:01.252733 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=2 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:28:01.578713 dv-cinder-852211-12-iscsi devstack@keystone.service[81590]: DEBUG dbcounter [-] [81590] Writing DB stats keystone:SELECT=40,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81590) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 12:32:04.989941 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533b71e8-7680-4465-ade6-7705c65c5b63 admin admin] Authenticating user token {{(pid=81591) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 12:32:04.991415 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-533b71e8-7680-4465-ade6-7705c65c5b63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:32:05.129177 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533b71e8-7680-4465-ade6-7705c65c5b63 admin admin] Validating token access rules against request {{(pid=81591) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 12:32:05.186614 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120cbaf0-f576-4c3e-b5b2-6bd5dce170a4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3adb2f0583c4ff782049dacdb3ee556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe863fcd75f1406ebbffc5016078f73e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81591) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 12:32:05.187259 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120cbaf0-f576-4c3e-b5b2-6bd5dce170a4 service placement] REQUEST_METHOD: `GET` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 12:32:05.187534 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120cbaf0-f576-4c3e-b5b2-6bd5dce170a4 service placement] SCRIPT_NAME: `/identity` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 12:32:05.187798 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120cbaf0-f576-4c3e-b5b2-6bd5dce170a4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81591) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 12:32:05.189516 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.fernet_utils [None req-120cbaf0-f576-4c3e-b5b2-6bd5dce170a4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81591) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 12:32:05.200331 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120cbaf0-f576-4c3e-b5b2-6bd5dce170a4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 12:32:05.202795 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120cbaf0-f576-4c3e-b5b2-6bd5dce170a4 service placement] RBAC: Authorization granted {{(pid=81591) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 12:32:05.212695 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: [pid: 81591|app: 0|req: 1477/2933] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 9 12:32:04 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 12:32:15.208332 dv-cinder-852211-12-iscsi devstack@keystone.service[81591]: DEBUG dbcounter [-] [81591] Writing DB stats keystone:SELECT=89 {{(pid=81591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}